CN115877723A - Home device control method, related device and system - Google Patents

Abstract

The present application discloses a method for controlling smart home equipment, a related device and a system. The method is based on the data banking system, wherein the data banking system includes: data collection, data processing, data opening and presentation of data value. In this method, the mobile terminal can collect relevant data such as the user's role in the family relationship, the space where the user is located, and set the user's authority to control the family device according to the data. Afterwards, the mobile terminal receives the user's operation, identifies the user's identity, and controls the home device to execute the corresponding function according to the authority corresponding to the user's identity. In addition, a rule is set for children and parents to use the mobile terminal. When the rule is broken, they can cooperate with other family devices to execute and complete the set tasks before continuing to use the mobile terminal. In addition, when the mobile terminal detects that the user is away from home, it can also control the home device to perform operations corresponding to the above schedule according to the collected user schedule data.

Description

Home equipment control method, related device and system

Technical Field

The present application relates to the field of communication technology, and in particular to a home device control method, related devices and systems.

Background Art

With the gradual development of information technology, the improvement of network technology, and the increasing abundance of applicable network carriers, more and more home devices have gradually entered every household, and more and more families have begun to pursue smart life. At that time, the concept of smart home was proposed. Smart home can also be called smart home service platform, which refers to the use of family affection as a bond, combined with new generation information technologies such as the Internet of Things, cloud computing, mobile Internet and big data, combined with automatic control technology, to effectively combine home life such as intelligent control of home devices, home environment perception, family health perception, home safety perception, information exchange, consumer services, etc., to create a healthy, safe, comfortable, low-carbon, convenient personalized home life.

How to control home devices and create a smart life for users is an urgent problem to be solved.

Summary of the invention

The present application provides a method, a related device and a system for controlling home devices. Home devices can be controlled according to the association between the user and the home space. In addition, when the user breaks the usage rules of the home device, it is necessary to coordinate with other home devices to complete the specified task. In addition, when the mobile terminal detects that the user has left home, it can also control the home device to perform operations corresponding to the events stored in the mobile terminal. This creates a more convenient, healthy, low-carbon and harmonious smart home.

In a first aspect, the present application provides a method for controlling a device, which is applied to a communication system comprising a first device and a second device; the second device is located in a first area in a first space; the method comprises: the first device receives a first operation; the first device identifies the identity of a user who inputs the first operation as a first user; the first device triggers the second device to start a first function; the first device receives the first operation again; the first device identifies the identity of a user who inputs the first operation as a second user; the first device refuses to trigger the second device to start the first function.

In the second aspect, the present application provides a device control method, which is applied to a communication system between a first device and a second device; the second device is located in a first area in a first space; the method includes: the first device receives a first operation, and the first operation is used to trigger the second device to start a first function; the first device identifies the identity of the user who inputs the first operation; if the user has the authority to control the devices in the first area, the first device triggers the second device to start the first function; if the user does not have the authority to control the devices in the first area, the first device refuses to trigger the second device to start the first function.

By implementing the method provided in the first aspect or the second aspect, the first device can set the user's control authority over the home device according to the relationship between the user and the home space, thereby ensuring that family members can only use the home devices within the scope of authority. It can also prevent people who do not have the authority to use home devices from performing some dangerous operations or malicious operations. For example, to prevent the elderly or children from using dangerous electrical appliances, or family visitors from using privacy devices in the home, such as expensive insurance, etc., which may cause losses to the family.

In combination with the method provided by the first aspect or the second aspect, in one embodiment, the above-mentioned communication system also includes a third device; before the first device receives the first operation, the method also includes: after the first device receives the second operation, it determines that the second device is located in the first area in the first space; or, the first device receives the first information sent by the third device, and the first information indicates that the second device is located in the first area in the first space.

In this way, the mobile terminal can divide the association between the room and the family members, and the association between the room and the home device according to the user operation, or the mobile terminal can automatically generate the permission for the family members to control the home device according to the association between the room and the family members, and the association between the room and the home device detected by other home devices. When the detection results of other home devices are wrong, resulting in the association between the room and the device finally generated by the mobile terminal to be wrong, the mobile terminal can also re-change the corresponding relationship between the room and the device according to the manual operation of the user. In this way, the mobile terminal can be implemented in a variety of ways, which increases the feasibility of this solution and increases the accuracy of the permission setting.

In combination with the method provided in the first aspect or the second aspect, the first information further indicates the type of the first area; the type includes: bedroom, kitchen, study, living room, bathroom, gym, and entertainment room.

In this way, the mobile terminal can divide the rooms contained in the home space into detailed parts, making it convenient for users to manage the devices in a certain room in a targeted manner.

In combination with the method provided in the second aspect, the type of the first area is determined by the third device according to the identified type of the second device.

In this way, the sweeping robot can obtain household devices when cleaning the house, and determine the type of rooms containing these devices based on the types of household devices.

In combination with the method provided in the second aspect, the second device includes the third device.

In combination with the method provided in the second aspect, the second device is a household device; and the third device is a sweeping robot.

In combination with the method provided in the second aspect, when the user does not have the authority to control the device in the first area, the first device outputs a first prompt message, and the first prompt message is used to prompt the user that he does not have the authority to control the second device to start the first function.

In this way, the mobile terminal can inform the user that the home devices outside the scope of the authority cannot be controlled, thereby avoiding the user from repeatedly inputting control operations.

In combination with the method provided in the second aspect, the communication system also includes a fourth device; when the user does not have the authority to control the device in the first area, the first device displays a first control; the first device detects the operation acting on the first control; the first device sends a first request to the fourth device, and the first request is used to request the user to obtain the authority to control the first function of the second device; the first device receives the authorization message sent by the fourth device, triggering the second device to start the first function.

In this way, when a family member, such as a child, does not have the authority to control a device, the member can send an authorization request to other family members, such as parents. When the parents ensure that the operation to be performed by the child is safe, they can authorize the child. This can ensure the safety of family members without affecting the use of family devices.

In combination with the method provided in the second aspect, after the first device refuses to trigger the second device to start the first function, the method also includes: the first device again identifies the identity of the user who inputs the first operation; if the user has the authority to control the devices in the first area, the first device triggers the second device to start the first function; if the user does not have the authority to control the devices in the first area, the first device refuses to trigger the second device to start the first function.

In this way, when the first device first recognizes that the current user does not have the authority to control the device, the first device can continue to authenticate the user's identity. During this process, assuming that the current user is accompanied by other family members such as parents, the parents can be requested to enter their identity information. In this way, the family device can continue to be controlled normally.

In combination with the method provided by the second aspect, before the first device triggers the second device to start the first function, the method also includes one or more of the following: the first device determines that there are other users in the first space; or, the first device determines that the distance between the user and the first space is less than a first value; or, the first device determines that the time of receiving the first operation is the user's non-working hours; or, the first device determines that the first function is used to reduce household power consumption.

In this way, mobile terminals can combine more factors to control home devices, ensuring the quality of family life in all aspects on the basis of low carbon and health.

In combination with the method provided in the second aspect, before the first device receives the first operation, the method also includes: the first device stores control permissions of multiple users for devices in various areas in the first space.

In combination with the method provided in the second aspect, the first space includes: home space and office space.

In this way, the control method of the device provided in this application can be used in home scenarios, office scenarios, or other scenarios.

In a third aspect, the present application provides a method for controlling a device, which is applied to a communication system of a fourth device and a fifth device; the method includes: the fourth device detects that the fourth device is outside a first electronic fence; the fourth device outputs a second prompt message, or the fourth device sends a second message to the fifth device, and the second information is used to indicate that the fourth device is outside the first electronic fence; the first electronic fence includes multiple closed areas and routes between the multiple closed areas.

In this way, when a user, such as a child, a family pet, or valuables, is lost and leaves a safe area, family members can be informed to protect the safety of family members, family pets, and valuables.

In combination with the method provided in the third aspect, the fourth device detects that the fourth device is located within the second electronic fence; the fourth device outputs third prompt information, or the fourth device sends third information to the fifth device, and the third information is used to indicate that the fourth device is located within the second electronic fence.

In this way, when a user, such as a child or a family pet, enters a dangerous place, family members can be informed to protect the safety of family members and family pets.

In combination with the method provided in the third aspect, the fourth device detects that the fourth device is outside the first electronic fence, specifically including: the fourth device detects that the fourth device is outside the multiple closed areas in the first electronic fence; or, the fourth device detects that the fourth device is outside the route in the first electronic fence for more than a first distance and/or is outside the route in the first electronic fence for more than a first time period.

In this way, the first device can set an electronic fence according to the user's operation or generate an electronic fence according to the user's daily activity trajectory.

In combination with the method provided in the third aspect, the fourth device detects that the fourth device is outside the first electronic fence. Before that, the method also includes: the fourth device sets the first electronic fence according to the third operation input by the user; or, the fourth device generates the first electronic fence according to the moving trajectory within the second time period, and the first electronic fence includes the moving trajectory.

In this way, the first device can set an electronic fence according to the user's operation or generate an electronic fence according to the user's daily activity trajectory.

In combination with the method provided in the third aspect, the boundary of the first electronic fence corresponds to the boundary of the area corresponding to the moving track, and the area enclosed by the first electronic fence is greater than or equal to the area corresponding to the moving track.

In combination with the method provided in the third aspect, the fifth device includes any one or more of the following: a mobile phone, a tablet, a smart watch, a bracelet, a household appliance or a car computer.

In this way, when the user leaves a safe area or enters a dangerous area, the devices at home, the devices carried by the user, and the vehicle can all sound alarms to ensure that other family members receive the prompt information as soon as possible.

In a fourth aspect, the present application provides a device control method, which is applied to a communication system of a first device, a second device and a fourth device; the method includes: the fourth device detects that the usage of the fourth device does not comply with the first rule; the fourth device stops providing part or all of the functions of the fourth device; the fourth device receives fourth information sent by the second device, and the fourth information indicates that the user has completed the first task; in response to receiving the fourth information, the fourth device continues to provide part or all of the functions of the fourth device; or, the fourth device receives fifth information sent by the first device, and the fifth information is used to instruct the fourth device to continue to provide part or all of the functions of the fourth device; in response to receiving the fifth information, the fourth device continues to provide part or all of the functions of the fourth device.

When implementing the method provided in the fourth aspect, when a user breaks the rules for using an electronic device, he or she needs to cooperate with other devices to complete the corresponding unlocking task before continuing to use the electronic device normally.

In combination with the method provided in the fourth aspect, before or after the fourth device stops providing part or all of the functions of the fourth device, the method also includes: the fourth device outputs a fourth prompt message, and the fourth prompt message is used to instruct the user to perform the first task.

In this way, after the device detects that the user has broken the rules, before or after locking some or all functions, the user can be prompted to perform the corresponding unlocking task to unlock the functions.

In combination with the method provided in the fourth aspect, before the fourth device detects that the usage of the fourth device does not comply with the first rule, the method also includes: the fourth device sets the first rule according to the fourth operation input by the user; and the fourth device receives the first rule sent by the first device.

In this way, the usage rules for a user to use a certain device can be set in the local device or remotely set in other devices.

In combination with the method provided in the fourth aspect, before or after the fourth device receives the fourth information sent by the second device, and before the fourth device continues to provide part or all of the functions of the fourth device; the method also includes: the second device sends the fourth information to the first device; the first device detects the fifth operation; the first device sends the fifth information to the fourth device, and the fifth information is used to instruct the fourth device to continue to provide part or all of the functions of the fourth device.

In this way, after other devices detect that the user has completed the unlocking task, the information of executing the unlocking task can be sent to the parent side. After the parent side reviews and approves it, the user can unlock the current device.

In combination with the method provided in the fourth aspect, after the fourth device receives the fourth information sent by the second device, and before the fourth device continues to provide part or all of the functions of the fourth device; the method also includes: the fourth device sends the third information to the first device; the first device detects the fifth operation, and the first device sends the fifth information to the fourth device, and the fifth information is used to instruct the fourth device to continue to provide part or all of the functions of the fourth device.

In this way, after other devices detect that the user has completed the unlocking task, they will send the information of executing the unlocking task to the device side that breaks the rule. The device can send the information of executing the unlocking task to the parent side. After the parent side reviews and approves it, the user can unlock the current device.

In combination with the method provided in the fourth aspect, the first rule includes any one or more of the following: the length of time that the fourth device is in an unlocked state every day is less than the first length of time; the length of time that the fourth device is continuously in an unlocked state is less than the second length of time; the fourth device is in a locked screen state at other times outside a third time period; the fourth device refuses to run applications other than the first application; the length of time that the fourth device continuously runs the first application is less than the third length of time.

In this way, multiple usage rules can be set to achieve all-round management of user usage rules.

In combination with the method provided in the fourth aspect, the first application includes: an instant messaging application, an education and learning application, a sports and health application, and an event memo application.

In combination with the method provided in the fourth aspect, the first application is a third-party application and/or a system application.

In combination with the method provided in the fourth aspect, the first task includes any one or more of the following: studying, exercising, resting or doing housework.

In this way, when users break the rules of using electronic devices, they can rest, exercise, learn and do housework. Such a combination of work and rest can ensure the physical and mental health of users.

In combination with the method provided in the fourth aspect, before the fourth device detects that the usage of the fourth device does not comply with the first rule, the method also includes: the fourth device outputs the first rule for prompting the user to use the fourth device.

In this way, users can be reminded of the usage rules in advance to prevent them from breaking the usage rules.

In combination with the method provided in the fourth aspect, the fourth device receives the fourth information sent by the second device, which specifically includes: the second device authenticates the identity of the user, and sends the third information to the fourth device when the identity of the user is the second user.

In this way, it can be ensured that when the user performs the unlocking task, it is he who completes the unlocking task, rather than being replaced by other people.

In combination with the method provided in the fourth aspect, the second user is a child.

This can help manage children’s use of devices and prevent them from becoming addicted to electronic devices or using them improperly.

In combination with the method provided in the fourth aspect, the first device is the mobile phone, tablet or notebook of the parent of the second user; the fourth device is the smart watch, bracelet, mobile phone, tablet or notebook of the second user; the second device is a device that provides sports services or learning services.

In a fifth aspect, the present application provides a method for controlling a device, which is applied to a communication system of a first device, a second device, a fourth device and a sixth device; the method includes: the first device detects a sixth operation, which is used to start a second function; the first device obtains the status of the second user according to a seventh operation input by the user, or obtains the status of the second user through the second device or the fourth device or the sixth device; when the status of the second user is one or more of the following: not sleeping, studying, the distance between the second user and the first user is less than the second distance and the preset distance, or the second user and the first user are in the same room; the first device refuses to start the second function.

By implementing the method provided in the fifth aspect, rules can be set for parents to use electronic devices, so that parents can efficiently accompany their children's growth and establish a good image for their children.

In a sixth aspect, the present application provides a method for controlling a device, which is applied to a communication system of a first device and a sixth device; the method includes: the first device detects an eighth operation input by the second user, and the eighth operation is used to start a third function; the first device obtains the status of the first user; when the status of the first user is one or more of the following: sleeping, away from home, working; the first device refuses to start the third function.

By implementing the method provided in the sixth aspect, rules can be set for children to use their parents' electronic devices, so that children can use electronic devices correctly with their parents' company and avoid making illegal operations.

In combination with the method provided in the fifth aspect or the sixth aspect, the status of the first user is set by the first device according to the operation input by the first user; or, the status of the first user is detected by the first device or the sixth device and sent to the first device.

In this way, parents can set their own status and then constrain themselves to use electronic devices according to regulations, so as to actively require themselves to accompany their children with high quality and establish a good image for their children. It is also possible for electronic devices to detect the status of the home to constrain parents to use electronic devices according to regulations.

In combination with the method provided in the fifth aspect or the sixth aspect, the second user is a child; and the first user is the parent of the second user.

In combination with the method provided in the fifth aspect or the sixth aspect, it is characterized in that the first device is a mobile phone, tablet or notebook of the first user; the sixth device includes any one or more of the following: a home monitoring device of the first user, a smart watch, bracelet, mobile phone, tablet or notebook of the second user.

In this way, the devices for detecting the user status may include multiple types, thereby improving the feasibility of the solution.

In the seventh aspect, the present application provides a device control method, which is applied to a communication system including a smart door lock, a first device and a second device; the method includes: the smart door lock detects a ninth operation, and the smart door lock opens the door; when the distance between the smart door lock and the first device gradually increases, the smart door lock performs one or more of the following: refusing to close the door, outputting a fifth prompt information for prompting the user to carry the first device, or triggering the first device and/or the second device to output a fifth prompt information for prompting the user to carry the first device.

After implementing the method provided in the seventh aspect, when the user leaves home and does not bring the electronic device, the user can be reminded to bring the electronic device, thereby avoiding the user wasting time leaving home and returning home, thereby delaying the trip.

In combination with the method provided in the seventh aspect, before the smart door lock detects the ninth operation, the method also includes: the smart door lock determines that the distance between the user and the smart door lock is less than a second value; the smart door lock outputs a fifth prompt information for prompting the user to carry the first device, or triggers the first device and/or the second device to output the fifth prompt information for prompting the user to carry the first device.

In this way, when the user is close to home and has the intention to leave home, the user can be reminded in advance to bring electronic devices to avoid wasting time leaving home and returning home, thus delaying the trip.

In combination with the method provided in the seventh aspect, the smart door lock, the first device and the second device are located in a first space; the method also includes: in a fourth time period, the first device determines that the user leaves the first space, and the user will carry the first item when traveling in the fourth time period; the first device outputs a sixth prompt message, and the sixth prompt message is used to prompt the user to carry the first item.

In this way, when the electronic device detects that the user needs to carry necessary items such as an ID card, notebook, etc. when traveling, it can remind the user to bring the relevant items when the user goes out, avoiding the user from wasting time leaving home and returning home, and delaying the trip.

In combination with the method provided in the seventh aspect, sixth information is stored in the first device, and the sixth information indicates an event in which the user travels with the first item during the fourth time period.

In this way, the electronic device can obtain information related to the user's travel events in advance.

In combination with the method provided in the seventh aspect, the smart door lock, the first device and the second device are located in a first space, and the method also includes: in a fifth time period, the first device determines that the user has left the first space; extreme weather occurs in the fifth time period; the first device outputs a seventh prompt message, and the seventh prompt message is used to prompt the user to bring protective equipment.

In this way, when the electronic device detects that the user is traveling, it can remind the user to bring the corresponding protective equipment according to the current weather conditions, so as to avoid the user wasting time leaving home and returning home, delaying the trip, and improving travel efficiency.

In combination with the method provided in the seventh aspect, the first device determines that the user leaves the first space, specifically including: the first device receives the seventh information sent by the smart door lock, the seventh information indicates that the user leaves the first space, or the first device detects that the user leaves the first space.

In this way, the electronic device can receive a message from other devices indicating that the user has left home, or detect that the user has left home by itself to determine the user's away-from-home status.

In combination with the method provided in the seventh aspect, the first device detects that the user leaves the first space, specifically including: the first device detects that the communication connection with the router in the home is disconnected; or, the first device detects that the communication connection with the second device is disconnected; or, the first device obtains the travel event recorded in the first device; or, the first device detects that the user takes a bus by using the NFC function or scanning a barcode; or, the first device detects that the distance between the user and the first space exceeds a third value.

In this way, the electronic device can detect that the user has left home in a variety of ways, thereby improving the feasibility of the solution.

In combination with the method provided in the seventh aspect, the communication system also includes a vehicle; the method also includes: the vehicle detects the tenth operation, and the door of the vehicle opens; when the distance between the first door and the first device gradually increases, the vehicle performs one or more of the following: refusing to close the door, outputting an eighth prompt information for prompting the user to carry the first device, or triggering the first device and/or the second device to output the eighth prompt information for prompting the user to carry the first device.

In this way, when the user gets off the bus and does not bring the electronic device, the user can be reminded to bring the electronic device to avoid wasting time and delaying the trip.

In combination with the method provided in the seventh aspect, the communication system also includes a vehicle; the method also includes: the vehicle determines that the distance between the user and the vehicle door is less than a fourth value; the vehicle outputs an eighth prompt information for prompting the user to carry the first device, or triggers the first device and/or the second device to output the eighth prompt information for prompting the user to carry the first device.

In this way, when the user approaches the car door and has the intention to get off the car, the user can be reminded in advance to bring the electronic device to avoid wasting time and delaying the trip.

In combination with the method provided in the seventh aspect, the communication system also includes a vehicle; the method also includes: the vehicle detects the eleventh operation, the vehicle locks the doors and windows; the vehicle detects that there is a user in the vehicle, and the vehicle performs one or more of the following: opens the window, opens the door, outputs the ninth prompt information for prompting that there is a user in the vehicle, or triggers the first device and/or the second device to output the ninth prompt information for prompting that there is a user in the vehicle.

In this way, users can be reminded that there are children or elderly people left in the car, and the windows and doors can be automatically opened to avoid safety accidents involving children or elderly people in the car.

In combination with the method provided in the seventh aspect, after the vehicle opens the window or door, the eighth information is sent to the first device, and the eighth information indicates that the vehicle has opened the window or door; the first device displays the second control, and the first device detects the operation acting on the second control, and the first device triggers the vehicle to lock the windows and doors.

In this way, when the user instructs to get off the vehicle briefly, or the children or the elderly in the vehicle have already gotten off the vehicle, in order to ensure the safety of valuables in the vehicle, the user can remotely control the vehicle to lock the windows and doors.

In an eighth aspect, the present application provides a method for controlling a device, which is applied to a communication system comprising a first device, a second device, and a vehicle; the method includes: in a sixth time period, the first device performs a twelfth operation; the user will perform a first event in the sixth time period; the twelfth operation includes any one or more of the following: the first device triggers the third device to start a fifth function, and the fifth function supports the user to execute the first event; or, the first device triggers the vehicle to start, and the vehicle start supports the user to execute the first event.

By implementing the method provided in the eighth aspect, the electronic device can control the home devices to perform corresponding operations based on the information of the events recorded in the electronic device, thereby reducing user operations and alleviating the user's memory burden.

In combination with the method provided in the eighth aspect, before the first device triggers the third device to execute the fifth function; or, before the first device triggers the vehicle to start, the method also includes: the first device outputs a tenth prompt message; the tenth prompt message is used to prompt the user that the third device is about to execute the fifth function, or, the vehicle is about to start.

In this way, the electronic device can notify the user in advance that it will control other devices to perform corresponding operations, thereby ensuring the user's right to know.

In combination with the method provided in the eighth aspect, after the first device executes the twelfth operation, the method also includes: the first device displays a third control, the first device detects an operation acting on the third control, and the first device withdraws the twelfth operation.

In this way, after automatically controlling other devices to execute corresponding events, the electronic device can also display corresponding controls for the user to withdraw the above control operations.

In the ninth aspect, the present application provides a method for controlling a device, which is applied to a fourth device; the method includes: the fourth device detects that the fourth device is outside a first electronic fence; the fourth device outputs a second prompt message, or the fourth device sends a second message to a fifth device, and the second information is used to indicate that the fourth device is outside the first electronic fence; the first electronic fence includes multiple closed areas and routes between the multiple closed areas.

In combination with the method provided in the ninth aspect, the fourth device detects that the fourth device is located within the second electronic fence; the fourth device outputs third prompt information, or the fourth device sends third information to the fifth device, and the third information is used to indicate that the fourth device is located within the second electronic fence.

In combination with the method provided in the ninth aspect, the fourth device detects that the fourth device is outside the first electronic fence, specifically including: the fourth device detects that the fourth device is outside the multiple closed areas in the first electronic fence; or, the fourth device detects that the fourth device is outside the route in the first electronic fence for more than a first distance and/or is outside the route in the first electronic fence for more than a first time period.

In combination with the method provided in the ninth aspect, the fourth device detects that the fourth device is outside the first electronic fence. Before that, the method also includes: the fourth device sets the first electronic fence according to a third operation input by the user; or, the fourth device generates the first electronic fence according to the moving trajectory within the second time period, and the first electronic fence includes the moving trajectory.

In combination with the method provided in the ninth aspect, the boundary of the first electronic fence corresponds to the boundary of the area corresponding to the moving trajectory, and the area enclosed by the first electronic fence is greater than or equal to the area corresponding to the moving trajectory.

In combination with the method provided in the ninth aspect, the fifth device includes any one or more of the following: a mobile phone, a tablet, a smart watch, a bracelet or a household device.

In the tenth aspect, the present application provides an example of a device control method, which is applied to a fourth device, and the method includes: the fourth device detects that the usage of the fourth device does not comply with the first rule; the fourth device stops providing part or all of the functions of the fourth device; the fourth device receives fourth information sent by the second device, and the fourth information indicates that the user has completed the first task; in response to receiving the fourth information, the fourth device continues to provide part or all of the functions of the fourth device; or, the fourth device receives fifth information sent by the first device, and the fifth information is used to instruct the fourth device to continue to provide part or all of the functions of the fourth device; in response to receiving the fifth information, the fourth device continues to provide part or all of the functions of the fourth device.

In combination with the method provided in the tenth aspect, before or after the fourth device stops providing part or all of the functions of the fourth device, the method also includes: the fourth device outputs a fourth prompt message, and the fourth prompt message is used to instruct the user to perform the first task.

In combination with the method provided in the tenth aspect, before the fourth device detects that the usage of the fourth device does not comply with the first rule, the method also includes: the fourth device sets the first rule according to the fourth operation input by the user; and the fourth device receives the first rule sent by the first device.

In combination with the method provided in the tenth aspect, after the fourth device receives the fourth information sent by the second device, and before the fourth device continues to provide part or all of the functions of the fourth device; the method also includes: the fourth device sends the third information to the first device; the fourth device receives the fifth information sent by the first device after detecting the fifth operation, and the fifth information is used to instruct the fourth device to continue to provide part or all of the functions of the fourth device.

In combination with the method provided in the tenth aspect, the first rule includes any one or more of the following: the length of time that the fourth device is in an unlocked state every day is less than a first length of time; the length of time that the fourth device is continuously in an unlocked state is less than a second length of time; the fourth device is in a locked screen state at other times outside a third time period; the fourth device refuses to run applications other than the first application; the length of time that the fourth device continuously runs the first application is less than a third length of time.

In combination with the method provided in the tenth aspect, the first application includes: an instant messaging application, an education and learning application, a sports and health application, and an event memo application.

In combination with the method provided in the tenth aspect, the first application is a third-party application and/or a system application.

In combination with the method provided in the tenth aspect, the first task includes any one or more of the following: studying, exercising, resting or doing housework.

In combination with the method provided in the tenth aspect, before the fourth device detects that the usage of the fourth device does not comply with the first rule, the method also includes: the fourth device outputs the first rule for prompting the user to use the fourth device.

In combination with the method provided in the tenth aspect, the fourth device receives the fourth information sent by the second device, specifically including: the fourth device receives the third information sent by the second device when authenticating the identity of the user as the second user.

In combination with the method provided in the tenth aspect, according to the method of claim 67, the second user is a child.

In combination with the method provided in the tenth aspect, the first device is the mobile phone, tablet or notebook of the parent of the second user; the fourth device is the smart watch, bracelet, mobile phone, tablet or notebook of the second user; the second device is a device that provides sports services or learning services.

In the eleventh aspect, the present application provides a method for controlling a device, which is applied to a first device; the method includes: the first device detects a sixth operation, which is used to start a second function; the first device obtains the status of a second user according to a seventh operation input by the user, or obtains the status of the second user through a sixth device; when the status of the second user is one or more of the following: not sleeping, studying, the distance between the second user and the first user is less than the second distance and the preset distance, or the second user and the first user are in the same room; the first device refuses to start the second function.

In the twelfth aspect, the present application provides a method for controlling a device, which is applied to a first device; the method includes: the first device detects an eighth operation input by the second user, and the eighth operation is used to start a third function; the first device obtains the status of the first user; when the status of the first user is one or more of the following: sleeping, away from home, working; the first device refuses to start the third function.

In combination with the method provided in the eleventh aspect or the twelfth aspect, the status of the first user is set by the first device according to the operation input by the first user; or, the status of the first user is detected by the first device or the sixth device and then sent to the first device.

In combination with the method provided in the eleventh aspect or the twelfth aspect, the second user is a child; and the first user is the parent of the second user.

In combination with the method provided in the eleventh aspect or the twelfth aspect, it is characterized in that the first device is a mobile phone, tablet or notebook of the first user; the sixth device includes any one or more of the following: a home monitoring device of the first user, a smart watch, bracelet, mobile phone, tablet or notebook of the second user.

In a thirteenth aspect, the present application provides an electronic device, the electronic device comprising one or more processors and one or more memories; wherein the one or more memories are coupled to the one or more processors, the one or more memories are used to store computer program code, the computer program code comprising computer instructions, when the one or more processors execute the computer instructions, the electronic device executes the method as described in any one of the eighth aspect, the ninth aspect, the tenth aspect and the eleventh aspect. In a thirteenth aspect, the present application provides a computer program product comprising instructions, when the computer program product is run on an electronic device, the electronic device executes the method as described in any one of the eighth aspect, the ninth aspect, the tenth aspect and the eleventh aspect.

In a fourteenth aspect, the present application provides a computer-readable storage medium comprising instructions, which, when executed on an electronic device, causes the electronic device to execute a method as described in any one of the eighth aspect, the ninth aspect, the tenth aspect and the eleventh aspect.

In a fifteenth aspect, the present application provides a communication system, which includes a first electronic device and a second electronic device, and the first electronic device is used to execute the method described in any one of the eighth aspect, the ninth aspect, the tenth aspect and the eleventh aspect.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1A is a management process for user data in a data bank provided by an embodiment of the present application;

FIG1B is a schematic diagram of a communication system provided in an embodiment of the present application;

2A-2B are diagrams of the software and hardware architecture of a mobile terminal provided in an embodiment of the present application;

FIG3 is a hardware architecture diagram of a home device provided in an embodiment of the present application;

FIG4 is a hardware architecture diagram of a home device provided in an embodiment of the present application;

FIG5A is a schematic diagram of family member relationships provided in an embodiment of the present application;

FIG5B is a schematic diagram of the authority levels respectively possessed by family members according to an embodiment of the present application;

FIG5C is a schematic diagram of a family floor plan structure provided in an embodiment of the present application;

FIG5D is a schematic diagram of a home space layout structure provided in an embodiment of the present application;

6A-6V exemplarily show a series of operation interface diagrams for a user to create a “smart home”;

7A to 7T are schematic diagrams of a series of operation interfaces for a user to set control permissions for family members;

8A-8E are schematic diagrams of an initialization operation interface for a user to set an identity authentication method;

8F-8X exemplarily illustrate user interfaces for family members to control home devices through the smart home APP;

9A-9G exemplarily show a series of user interface schematic diagrams after a mobile terminal detects a sensitive operation;

9H-9M exemplarily show a series of user interface schematic diagrams after the mobile terminal detects another sensitive operation;

9N-9Q exemplarily show a series of user interface schematic diagrams after the mobile terminal detects another sensitive operation;

9R-9W exemplarily illustrate a series of user interface schematic diagrams after the mobile terminal detects another sensitive operation;

9X-9Y exemplarily show a series of user interface schematic diagrams after the mobile terminal detects another sensitive operation;

10A-10H exemplarily show a series of operation interface diagrams for setting usage rules and related tasks;

11A-11R exemplarily show a series of operation interface schematic diagrams when the usage rules are broken;

11S to 11X exemplarily illustrate a series of operation interface diagrams when a user performs an unlocking task;

11Y-11Z-5 exemplarily show a series of operation interface schematic diagrams when a user performs another unlocking task;

12A-12V exemplarily show a series of user interface schematic diagrams of a mobile terminal actively detecting a user status;

12W to 12Z exemplarily show a series of operation interface schematic diagrams when a user inputs a user status to a mobile terminal;

13A-13C exemplarily illustrate schematic diagrams of user interfaces for a mobile terminal to control home devices according to a user status;

13D-13J exemplarily illustrate schematic diagrams of user interfaces in which a mobile terminal controls home devices according to a user state and acquired event information;

13K-13L exemplarily illustrate schematic diagrams of user interfaces in which a mobile terminal controls home devices based solely on acquired events.

DETAILED DESCRIPTION

The technical solutions in the embodiments of the present application will be described clearly and in detail below in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, "/" means or, for example, A/B can mean A or B; "and/or" in the text is only a description of the association relationship of the associated objects, indicating that there can be three relationships, for example, A and/or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more.

Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

The term "user interface (UI)" in the following embodiments of the present application refers to a medium interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The user interface is a source code written in a specific computer language such as Java and extensible markup language (XML). The interface source code is parsed and rendered on an electronic device, and finally presented as content that the user can recognize. The commonly used form of user interface is a graphical user interface (GUI), which refers to a user interface related to computer operations that is displayed in a graphical manner. It can be a visual interface element such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc. displayed on the display screen of an electronic device.

With the increasing richness of communication means and the accelerated development of the digitization of the physical world, users enjoy more and more digital services, and users generate more and more data in their lives and work. User data is large in scale, diverse in structure and growing rapidly. In addition, user data has great value, such as being used to build user attributes, improve user experience, and develop new application technologies. User attributes can also be called user portraits.

Sharing and opening up user data can effectively connect various types of information, tap into the value contained in user data, and bring more convenient services to users' lives. However, sharing and opening up user data also carries the risk of leaking user privacy information, causing users to worry about the security of their personal data. In actual applications, out of privacy protection, users are usually reluctant to open up their own user data, which creates obstacles to tapping into the value of user data.

How to share and open user data under the premise of reasonableness and compliance, maximize the value of user data, fully tap the value of data, and ensure that user privacy is not leaked is the current and future research direction and also a concern of users.

The embodiment of the present application provides a management mechanism for a data bank, which can protect user privacy from being leaked while sharing and opening user data. The data bank has the characteristics of full life cycle governance and control, data leakage prevention and tampering prevention, participant identity authentication and access control, and value sharing. Specifically, the data bank ensures the orderly and controllable storage, sharing, application and destruction of data in the digital world during the life cycle, and ensures that all participants such as data providers, data processors, data trading platforms and data consumers can obtain benefits. The data bank promotes data to become the foundation of the prosperity of the digital economy and meets the demand for data to become a production factor of the market economy. The data bank provides technical support by marking the entire life cycle of data, authenticating and controlling the identities of all parties involved in the digital economy, and tracking and recording the flow of data throughout the process, to prevent information leakage and tampering during the data life cycle.

A data bank is a trusted institution that manages user data. It is responsible for the storage, sharing, distribution, withdrawal, tracing, and use of user data. It can also be used to manage the life cycle of user data, i.e., the validity period. A data bank can manage user data with user authorization and pay the owner of the user data.

Data banks use the interconnection between devices and each device's understanding of other devices' capabilities to store the generated data in an orderly manner by category. When storing data, the data is tagged with relevant tags, such as time, location, task, event, and then stored in a manner similar to a relational database to facilitate subsequent search or use of the data. User data can be stored in units of individuals, families, companies, or other groups. When storing data, it can also be classified and graded according to its importance and sensitivity, and the user can set access rights for each type of data, so that it can be guaranteed not to be leaked or abused during the data's life cycle.

When users store data in a data bank, they can encrypt the data and set access policies. Other users can only view the data stored in the data bank if they meet the access policies. Encryption technologies include but are not limited to attribute-based encryption (ABE) algorithms.

User data can also be stored in the data bank after being desensitized. For example, data that is strongly related to user privacy, including the user's name, address, contact information, etc., can be deleted before being stored in the data bank.

The embodiment of the present application also provides a possible desensitization processing scheme based on an intermediate server. The intermediate server can obtain the user data uploaded by the user through the end-side device, and provide the desensitized or blurred user data to the service provider based on the user data (such as a server providing medical and health services, a server providing traffic accident analysis, a server providing advertising recommendations, etc.), and does not directly publish all the user data to the service provider to ensure that the user data is not leaked or abused, and to protect the user's privacy. For example, after the user uploads the user data to the intermediate server through the end-side device, the intermediate server binds the user's real identity information (such as name, contact information, address, etc.) to the user data. When a service provider based on the user data requests the user data, the intermediate server can bind the user data and the user's virtualized identity information and send it to the service provider, and store the association relationship between the virtualized identity information and the real identity information. In this way, the service provider will not know the relationship between the user data and the real user identity, and the leakage of user sensitive data and privacy is avoided. Among them, the intermediate server can also be called an intermediate agent, a proxy server, a relay server, and other names.

Data banks can obtain massive amounts of user data, and with the user's consent, combine massive amounts of user data from all walks of life and all types of users, use these user data to perform more operations, tap the economic value and social value of these user data, and promote the development of various industries. If the user stops authorizing later, the data bank will stop the corresponding operation. For example, after the user stops authorizing, the data bank can clean up the corresponding data and will not back up or copy the data. This ensures that user data is used reasonably and orderly, and can also protect user privacy.

The data bank integrates the data encryption system, key management, and access control strategy into the data itself in the form of data tags, so that data can still be consistently protected when flowing across physical nodes.

The data bank itself can also be implemented as a multi-level or multi-category structure, and data banks at different levels have different permissions. For example, a data bank can include a general data bank and a sub-data bank. For another example, a data bank can include a data bank for managing sports and health data, a data bank for managing travel data, and so on.

It can be seen that user data is like currency, and a data bank is like a money bank. The management of user data by a data bank is similar to the management of currency by a money bank.

Data bank is just a term used in the embodiment of the present application, and its meaning has been recorded in the embodiment, and its name does not constitute any limitation to the embodiment. In some other embodiments, "data bank" may also be called other terms such as "data bank".

The data bank provided in the embodiment of the present application is introduced below in conjunction with FIG. 1A .

Referring to FIG. 1A , FIG. 1A exemplarily shows a process for managing user data in a data bank provided by an embodiment of the present application.

As shown in Figure 1A, the anonymized smart service data application platform shown in Figure 1A is the intermediate server mentioned above, and the various organizations shown in Figure 1A are the user data-based service providers mentioned above, such as organizations that provide medical and health services, organizations that provide traffic accident analysis, organizations that provide advertising recommendations, and so on.

User data may include but is not limited to the following types: sports and health data, travel data, habit data, device data, etc.

Sports health data is used to characterize the user's physical condition. Sports health data may include age, gender, height, weight, blood pressure, blood sugar, blood oxygen, respiratory rate, heart rate, ECG waveform, body fat percentage, body temperature and other physiological data.

Travel data represents the travel plans that users have executed or are about to execute, as well as the actual travel situation. Travel data may include: order data (such as air tickets, bus tickets, hotels, takeaway orders, etc.), schedule data, memo data, etc. Travel data may also include: vehicle information (such as speed, fuel level, navigation information, driving mode, etc.), road conditions, road infrastructure information (such as traffic light signals), historical travel records, etc.

Habit data: refers to data related to the user's living habits, including sleep data and activity area related data.

Device data represents the configuration of the electronic device used by the user, as well as the connection between the electronic device and other devices. Device data may include, for example: the list of applications (APPs) installed in the electronic device, the device model and configuration, the login account and password of the operating system (OS), the login account and password of each APP, the type, quantity and other information of other devices (such as smart home devices) connected to the electronic device, etc.

User data is not limited to the several types of user data listed above, and may also include more content, such as bank data, financial data, personal identity information, social relationships, education, contact information, etc.

User data can be in the form of symbols, text, numbers, voice, images, videos, etc. User data can be digital information or paper information.

Based on the data bank management mechanism shown in FIG1A , the user data processing process mainly includes the following steps:

1. Collection of User Data

There are many ways to collect user data, which are not specifically limited here.

The methods of collecting user data may include but are not limited to the following:

(1) Various devices collect user data

User data can be collected through various devices. These devices can be interconnected and form an ecological chain, or they can be independent and unconnected.

For example, medical devices, wearable devices, or sports equipment can collect sports health data. A sphygmomanometer can collect blood pressure, a blood glucose meter can collect blood sugar, an oximeter can collect blood oxygen saturation and pulse rate, a thermometer can collect body temperature, an ECG recorder can collect ECG waveforms, a body fat scale can collect body fat percentage, and a smart watch or smart bracelet can collect heart rate, respiratory rate, blood oxygen, etc.

For example, mobile terminals, vehicle-mounted devices or smart door locks can collect travel data. Vehicle-mounted devices can collect vehicle speed and fuel level through sensors, and can also obtain current navigation information and driving mode. They can also obtain road infrastructure information through Bluetooth, cellular and other communication technologies, etc.

Another example: mobile terminals and home monitoring can collect data related to users' living habits. Mobile terminals can collect users' daily routines and frequently visited areas.

(2) Applications (APPs) collect user data

Various types of apps can be installed in electronic devices, including system apps and third-party apps. System apps refer to apps provided or developed by the manufacturer of the electronic device, and third-party apps refer to apps provided or developed by non-electronic device manufacturers. The manufacturer of an electronic device may include the manufacturer, supplier, provider or operator of the electronic device.

When an electronic device runs an APP to provide services to users, the APP can collect relevant user data. For example, travel data can be collected by smartphones, tablets and other devices through the corresponding APP. Among them, order data can be collected from life services such as ticket booking APPs, schedule data can be collected from schedule APPs, memo data can be collected from memos, and weather data can be collected from weather forecast applications. For example, takeaway APPs can collect users' dietary preferences, and e-commerce shopping APPs can collect users' shopping preferences.

(3) The OS of electronic devices collects user data

等等。OS is the most basic system software, which is used to manage the hardware, software and data resources of electronic equipment, control program operation, improve the human-computer interface, provide support for other software, etc. OS includes but is not limited to

etc.

When an electronic device runs an OS to provide services to users, the OS can collect relevant user data.

For example, the OS of electronic devices such as smartphones and wearable devices can collect user habit data. The OS of smartphones and tablets can monitor and learn the user's behavioral habits when using electronic devices such as smartphones and tablets, such as the user's commonly used input method, message mode, operation mode, etc., and generate habit data that characterizes the user's behavioral habits. Wearable devices such as smart bracelets can collect the user's hand shaking frequency, cadence, sleep and other habit data.

The OS of the electronic device can also collect device data.

For example, the OS of devices such as smartphones can obtain the list of apps installed on the device, the model and other configuration information, the OS login account and password, the login account and password of each app, the user's personal information and contact list, and can also obtain information about other devices connected to the device (such as type, quantity, location), etc. based on the connection status.

For another example, users can also directly set their frequently used operation modes and message modes on devices such as smart phones and tablet computers, and the OS of the device can collect the user's behavioral habits based on the user's settings.

(4) Information input devices of electronic devices collect user data

Information input devices may include but are not limited to display screens, cameras, microphones, mice, keyboards, and the like.

For example, users can input sports health data in the form of text, voice, etc., and the data can be received by devices such as smartphones and tablets through display screens, microphones, etc. For another example, electronic devices can also collect users' input method information and search records through display screens. For another example, smartphones and tablets can also scan paper medical examination reports, CT images, drug prescriptions, exercise prescriptions, etc. through cameras to obtain information in medical examination reports.

(5) Third-party organizations collect user data

The third-party institution may include a trusted institution that generates data, such as a hospital, a functional agency, an airline, etc. A trusted institution may be used to collect user data.

For example, hospitals can collect users' sports and health data. After a user visits a medical institution, the diagnosis results can be uploaded to the server provided by the medical institution, and then the user's smartphone, tablet computer and other data can download the diagnosis results from the server.

For example, government agencies may collect users' personal information, social relationships, contact information, etc.

For another example, an airline company may collect user's personal information, travel data, etc. The above-mentioned data collection methods are not limited. In some embodiments, user data may be collected in other ways, which are not limited here.

When collecting user data, on the one hand, the validity of the user data can be verified, invalid data such as duplication, redundancy, errors and garbled characters can be marked, and the data can be required to be resubmitted or re-entered; on the other hand, the integrity of the data can also be verified.

After collecting user data, you can also pre-process the user data. For example, standardize various types of data and classify them according to the importance, sensitivity, timeliness or other specific characteristics of the data to establish a unified data storage model. In this way, user data obtained through different methods and platforms can be stored with a unified standard, and complete and comprehensive data can be collected to facilitate the subsequent construction of complete user attributes. Here, different types of user data can have different storage standards. For example, sports health data and travel data can be stored with different standards.

2. Processing of User Data

Processing user data can provide better services to users and meet their actual needs.

With the user's consent, user data can be uploaded to an anonymization management platform to ensure that the information cannot be traced to an individual and to protect user privacy. At the same time, based on a variety of encryption technologies, such as secret computing and confidential computing, end-cloud joint reasoning is performed to form a complete attribute of the user and protect privacy data from leakage. Of course, different categories of user data can be uploaded to different anonymization management platforms, such as sports health data can be uploaded to the medical management platform, travel data can be uploaded to the travel management platform and the smart home management platform, etc.

The processing of user data can be applied to the following areas:

Smart Home

Electronic devices in a family can be networked to obtain an interconnected communication system, and a family includes many family members, such as the head of the family, the head of the family's parents (elderly), the head of the family's children (children), and sometimes family members also include domestic service personnel, tutors, family fitness coaches, etc. The electronic device can divide the family activity space for the above family members, that is, establish a corresponding relationship between the user's identity and the family space. When users of different identities use an electronic device in the home, the electronic device can analyze the collected user information, identify the user's identity, and then open the corresponding home appliance usage rights for the user. Among them, the user's information includes: the user's facial image, fingerprint, voice, posture, operating habits, etc. In addition, the electronic device can also limit the user's partial usage rights based on the user's identity information, the user's common activity space in the home, the user's behavior trajectory, the current time and other information.

The electronic device can detect whether the user has broken the usage rules of the device according to the set usage rules. When the electronic device detects that the usage rules of the device have been broken, the electronic device can instruct the user to complete the specified task according to the pre-specified task to continue to use the electronic device normally. The specified task can be completed in any one or more devices of the local electronic device and other home devices connected to the local electronic device, such as a learning machine, a sports bracelet, a treadmill, a sweeping robot, and a smart washing machine. The electronic device can analyze whether the user has completed the specified task based on the collected data related to the user's task execution. If so, the user is allowed to continue to use the electronic device normally. If not, the user is restricted from using some functions of the electronic device or locking the electronic device. The electronic device will also judge the user's status based on the acquired user data and device data. When the user is away from home, the electronic device will control the home device to execute the corresponding event in combination with the event information recorded in the application. The above-mentioned user data may include information about events stored by the user in the application, and the above-mentioned device data includes data related to the connection status of the device and other home devices. The event information recorded in the above-mentioned application may be travel records, weather information, to-do events, etc. The above-mentioned process of processing or processing user data can be performed by the electronic device on the end side, or by the server on the cloud side, or by the end-cloud combination. When processing user data, models can be formulated through preset simple rules, or models can be extracted through artificial intelligence (AI) engines, and user data can be processed or processed through the models. Preset rules can come from empirical knowledge or industry standards. These models can be used to construct user attributes based on user data, thereby providing targeted services to users.

It can be seen that processing user data can also make users' family life smarter, more convenient and safer.

3. Openness of user data

User data can be opened across devices to other devices, across applications to other applications, across OS to other OS, and across users to other users. Opening up user data allows electronic devices to integrate multiple user data, thereby providing users with more intelligent and convenient services. By opening up user data, massive amounts of user data no longer exist independently, but can be circulated and shared.

When opening user data, you can formulate corresponding permission control policies for using user data, and implement strict permission management and access control based on encryption algorithms to ensure that data is only opened when the recipient meets the access policy. In addition, you can also control system permissions such as data forwarding, screenshots, and printing. This can not only provide better services by opening user data, but also ensure the security of user data and avoid data leakage. For example, in the medical field, it can ensure the orderly sharing, controlled distribution, controllable withdrawal, and post-event traceability of medical data.

4. Presentation of the value of user data

User data can be displayed to users intuitively, allowing users to directly obtain user data collected by electronic devices, thereby understanding their own physical condition, the physical condition of family members, the status of home appliances in the home, and so on.

For example, the electronic device can obtain information such as the operating status of other devices in the smart home and present it to the user. The electronic device can generate the user's electronic fence based on the user's life trajectory related data and present it to the user to remind the user not to exceed the protection range of the electronic fence. When the user crosses the electronic fence, the electronic device is triggered to sound an alarm or send an alarm message to other home devices. In addition, the electronic device will also present a prompt message showing the usage rules of the electronic device to the user, and then it can determine whether the user breaks the usage rules of the electronic device based on the user's data of using the electronic device. If so, the prompt message is output to prompt the user to complete the set task to continue to use the electronic device normally. Finally, the electronic device can also control the home device to perform corresponding operations according to the to-do event based on the acquired user to-do event information, and present the corresponding prompt message to the user. Regarding the value presentation of user data, please refer to the relevant description of the UI implementation example below, which will not be repeated here.

User data can also be used to build user attributes for each user in order to provide more personalized services for the user. Here, different types of data can be used to build different categories of attributes for the same user, such as using sports health data to build sports health attributes, using travel data to build travel preference attributes, using data related to home life to build home preference attributes, using data related to entertainment life to build entertainment preference attributes, and so on. User attributes can be provided to the user for their own viewing, or provided to a third-party APP or third-party platform so that the third-party APP or third-party platform can provide personalized services to the user.

User data can also be provided to third-party apps or platforms, such as medical institutions, sports and health platforms, etc., and integrated to provide better services to users.

For example, third-party apps or platforms can develop AI medical services based on a large amount of medical data, such as medical imaging, auxiliary diagnosis, drug development, health management, gene sequencing, etc., which can promote the technological development of the medical industry. For another example, patients can get previous electronic medical records and examination reports from different hospitals to assist doctors in diagnosis, that is, medical data can flow safely between different medical institutions, improving the efficiency of patients' medical treatment. For another example, users can also conduct daily health monitoring management or chronic disease risk analysis based on the various physical examination data they upload daily, without worrying about their data being leaked.

In addition, users can also sell their own user data. In this way, with the permission of users, platforms with demand can obtain a large amount of effective user data and use this user data to develop richer application technologies and provide better services to users. In addition, users can also get benefits, achieving a win-win situation.

In some embodiments, different applications may be installed on the user's terminal device to support the collection, processing, opening and data presentation of user data in the different fields mentioned above. For example, a sports and health APP may be installed in the terminal device to support the relevant processing of sports and health data. For example, a smart travel APP may be installed in the terminal device to support the relevant processing of travel data. For example, a smart home APP may be installed in the terminal device to support the relevant processing of home data. For example, a smart life APP may be installed in the terminal device to support the relevant processing of life data.

The data bank management mechanism provided in the embodiment of the present application allows users to share and open user data. Under the premise of ensuring user privacy and security, the data bank can be circulated like an asset, fully tapping the value of user data, which can not only meet the actual needs of users, allowing users to live a healthy life and travel smartly, but also promote the healthy and prosperous development of various industries.

With the introduction of the concept of smart home, smart home applications are also developing rapidly. Smart home is a comprehensive use of the Internet of Things, cloud computing, mobile Internet and big data technologies, combined with automatic control technology, to effectively combine home life such as intelligent control of home devices, home environment perception, family health perception, home safety perception, information exchange, consumer services, etc., to create a healthy, safe, comfortable, low-carbon, convenient and personalized home life. Today, smart home has far exceeded the scope of smart home. Smart home includes smart home appliances, mobile terminals, travel equipment, etc., and the above devices can be collectively referred to as home devices. With intelligent control as the core of smart home, smart home provides platform-level support for social public service, social management, home energy efficiency and energy saving management, and commercial application services, greatly facilitating home life and is the cornerstone of future smart cities and smart society.

In some smart home application scenarios, on the one hand, users can use mobile terminals to perform linkage control on home devices. For example, before going to bed, users can initiate linkage control information for turning on the "sleep mode", thereby controlling the lights, curtains, televisions, stereos, and other devices that affect the user's sleep quality in the home to stop working. Usually, the linkage control information will control all home devices. For example, if one of the family members wants to go to bed and initiates linkage control information for turning on the "sleep mode", the lights, curtains, televisions, stereos, and other devices in all rooms of the home will stop working. If at this time, there are other family members who are washing, working, studying, exercising, doing housework, etc., then such a way of controlling home devices will seriously affect the lives of other family members and the relationships between family members. On the other hand, the home owner can grant others the permission to use home devices by sharing devices or adding family members. However, sharing devices requires sharing each device in the home, which is too cumbersome. Granting them the permission to use home devices by simply adding members simplifies the authorization operation, but it will allow the added members to have the permission to use all devices in the home. This will cause the private devices of family members to be used by other members, which will affect the security of home devices to a certain extent.

In other smart home application scenarios, you can set rules for a certain home device used by the user. When the user breaks the rules for using the home device, the home device will be locked until the authorization of other users is received before the home device can be unlocked to continue normal use. For example, in order to educate their children, the elders in the family will use the above method to set the usage time of their children's mobile phones, tablets, computers and other home devices. When the children break the rules for using the home device, the home device will be locked immediately. The parents must enter fingerprints, faces and other information into the home device, or the parents must send authorization information such as verification codes to their children's home devices through their home devices to unlock the home device. However, parents are usually busy with work and may not be able to unlock their children in time. At this time, the children may have other necessary situations to use their mobile phones; or when parents unlock their children, it will also affect the parents' work progress, etc., and thus affect the harmony of family relationships. It can be seen that the above method violates the creation of a healthy, comfortable and convenient personalized home life.

In other smart home application scenarios, when the mobile terminal detects that the user has left home, it can control the home devices to work together and execute the corresponding events in the memo based on the log information in the user's memo. However, the mobile terminal usually determines whether the user has left home through the message feedback from the smart door lock. But sometimes, the user just goes downstairs to throw away the garbage or goes to the convenience store to buy something and returns home soon. In this case, if the air conditioner, purifier, or kitchen appliances in the home are automatically powered off, the repeated switching of home devices will consume more power. It can be seen that the above method violates the creation of a healthy, comfortable, and convenient personalized home life.

In order to solve the technical problems existing in the above method, the embodiment of the present application provides an intelligent control method for home devices, and the intelligent control method for home devices includes the following points:

1. Family members’ control rights over home devices.

A family usually includes multiple members, and the roles played by these family members in family relationships and the spaces where they often move around in the family environment are usually different. Specifically, the roles of family members in family relationships include: parents, grandparents, children, domestic service personnel, tutors, private trainers and other family visitors, etc. Usually, parents in a family are the head of the family and are responsible for more family affairs; grandparents are usually older and have difficulty in moving; children are usually children who are receiving education. The spaces where family members often move around in the family environment can be specifically divided into: parents live in bedroom A, grandparents live in bedroom B, and children live in bedroom C. Other spaces where grandparents or parents often move around may also include kitchens, and spaces where parents and children often move around may also include study rooms, and other public spaces, such as living rooms, dining rooms, bathrooms (not bathrooms in bedrooms), entertainment rooms, gyms, etc. may be spaces where all members often move around.

Accordingly, the home devices deployed in different spaces in the home should be controlled by the owner of the space, and not all family members have the authority to control all home devices in the home. However, in the way that some family members set the control authority of home devices, the father and mother in a family, as the head of the family, should have the highest authority. In other words, in addition to the right to control the home devices in his own space, the head of the family also has the right to control the home devices in the spaces of other family members, such as children, that is, the right to control all home devices. In this way, a harmonious family can be created. The so-called harmony means that family members have certain relative authority to control home devices, and the head of the family has absolute authority to control home devices. This relative and absolute unity can improve the security of home devices and protect the personal habits or privacy of family members.

Therefore, the present application provides a method for controlling home devices, in which a mobile terminal can set the user's authority to control home devices according to factors such as the user's role in the family relationship, the user's space in the home space, etc. Afterwards, the mobile terminal receives the user's operation for controlling the home device, identifies the user's identity, and controls the home device to perform the corresponding function according to the authority corresponding to the user's identity.

2. How family members can control other members’ use of home device rules.

In the control method of the home device, the user can set the use rules of the home device according to the education plan for the children. If the children break the rules, they must complete the specified tasks in the corresponding home device, so as to achieve the education of the children.

The above usage rules may be the usage rules that parents set remotely for their children's mobile terminals in their own mobile terminals, or the usage rules may be set directly in their children's mobile terminals, which is not limited in the embodiments of the present application.

The above tasks can be completed through a mobile terminal with set usage permissions or other household devices in the home. For example, parents set the usage rules of their children's mobile terminals to 4 hours of usage per day, and set the unlocking tasks after breaking the rules to study for 1 hour and exercise for 1 hour. When the mobile terminal detects that the child breaks the above rules, the mobile terminal may lock the screen or lock some functions until the mobile terminal detects that the child uses the mobile terminal to complete the above tasks, or the mobile terminal receives relevant data that the child uses other household devices such as a treadmill to complete a 1-hour exercise task, and receives relevant data that the child uses other household devices such as a learning machine to complete a 1-hour learning task. Only then will the mobile terminal unlock the screen or unlock some functions. For details, please refer to the detailed description below, which will not be repeated here.

3. Control home devices based on the status of family members and/or events recorded by terminal devices.

In the control method of the home device, the mobile terminal can obtain the user's status information, use the corresponding smart scene mode according to the user's status, and control the home device according to the control strategy set in the smart scene mode; for example, if the mobile terminal detects that the user is sleeping, the sleeping mode provided in the smart home APP will be applied, and the home device will be controlled according to the control strategy set in the sleeping mode, such as turning off the bedroom lights, curtains, TV, etc. The introduction of the smart home APP can be referred to the detailed description below, which will not be repeated here.

In the control method of the home device, the mobile terminal can also obtain information about events recorded in the application, and control the home device to perform operations corresponding to the above events based on the detected information about events recorded in the application; for example, if it is detected that the user needs to attend an online meeting, the mobile terminal can control the study light to turn on before the meeting starts, and start the user's computer or notebook, etc.

In the control method of the home device, after the mobile terminal detects the user status and event information, if the status is a specified status, the mobile terminal controls the home device to perform a corresponding operation according to the detected event information. The relationship between the above-mentioned specified status and event can be set according to whether the user status and event have an impact on the user's life and family environment. For example, if the specified status is leaving home, the detected event is, for example, carrying documents recorded in the memo, or when there is a bad weather event recorded in the weather forecast, the mobile terminal reminds the user to bring documents when going out, close the windows and bring rain gear, etc. according to the event.

In this way, intelligent management, control and use of home devices can be achieved without user operation, thus creating a healthy, safe, comfortable, low-carbon and convenient personalized home life.

The above user statuses include: leaving home, returning home, sleeping, studying at home, working at home, exercising at home, getting off the car, etc.

The above-mentioned multiple applications can be: event memo applications such as calendars, alarm clocks, notes, memos, etc.; or life service applications such as applications for making appointments or purchasing medical treatment, dining, travel, and hairdressing services; or applications that provide public accounts and mini-program services, which can provide users with appointments or purchase medical treatment, dining, travel, hairdressing services, etc.; or weather forecast applications, which can provide users with weather forecast services; or online conference applications, which can provide users with meeting scheduling services; or online course learning applications, which can only allow users to book or purchase a course and then conduct online learning. The above-mentioned applications can be system applications or third-party applications. The embodiments of the present application are not limited to this.

In order to introduce the prompt method provided in the embodiment of the present application more clearly and in detail, the communication system provided in the embodiment of the present application is first introduced below.

FIG1B exemplarily shows a communication system 10 provided in an embodiment of the present application.

As shown in FIG1B , the communication system 10 may include multiple home devices 200 and a server 300. The home devices 200 include one or more smart home devices, smart home appliances, smart travel devices, and one or more mobile terminals 100 equipped by family members. The server 300 establishes a communication connection with one or more home devices 200.

或者其他操作系统的便携式电子设备。还应当理解的是，在本申请其他一些实施例中，移动终端100也可以是非便携式电子设备，例如汽车中安装的车载电脑、笔记本、具有触敏表面(例如触控面板)的膝上型计算机(laptop)、台式计算机等等。The mobile terminal 100 refers to a portable electronic device carried by a user, such as a smart phone, a smart watch, a tablet computer, a personal digital assistant (PDA), a smart wearable device, etc. Exemplary embodiments of portable electronic devices include but are not limited to

Or other portable electronic devices with operating systems. It should also be understood that in some other embodiments of the present application, the mobile terminal 100 may also be a non-portable electronic device, such as a vehicle-mounted computer installed in a car, a notebook, a laptop computer with a touch-sensitive surface (such as a touch panel), a desktop computer, etc.

In some embodiments of the present application, the mobile terminal 100 is installed with a smart home APP, and the control of home devices is achieved through the APP. Specifically, the mobile terminal 100 can set the user's authority to control home devices according to the user's identity and the user's space in the home. When the mobile terminal 100 detects the user's operation to control the home device, it can first identify the user's identity and determine whether the user has the authority to control the home device. When the device is deployed in the user's space, the mobile terminal executes the corresponding event according to the user's operation. Please refer to the detailed description below, and the embodiments of the present application will not be repeated here.

In other embodiments of the present application, the user can set the usage rules of the mobile terminal 100, and set the unlocking task when the rules are broken. When the mobile terminal 100 detects that the user breaks the rules, the mobile terminal 100 will lock the screen or lock some functions. When it is detected that the user completes the corresponding unlocking task; or receives the relevant data of the user performing the unlocking task sent by other home devices, and determines that the user completes the relevant task, the mobile terminal 100 will unlock. Please refer to the detailed description below, and the embodiments of the present application will not be repeated here.

In other embodiments of the present application, when the mobile terminal 100 detects that the user is away from home, it can control the home device to perform corresponding operations according to the events recorded in the acquired application, such as reminding the user to bring items. In addition, the mobile terminal 100 can also detect the user's status, such as leaving home, returning home, sleeping, studying at home, working at home, etc., and apply the smart scene mode provided by the smart home APP according to the user's status. Please refer to the detailed description below, and the embodiments of the present application will not be repeated here.

In an embodiment of the present application, an application (Application, App) can be installed in the mobile terminal 100. For example, a user can install a "smart home" application in the mobile terminal 100. The user can interact with the server 300 through the "smart home" application and control the home device 200 through the server 300.

It is understandable that the above-mentioned "smart home" application can be a third-party application, a system application, a system service component installed by the electronic device, or a small program that can be called without installation, etc., and the embodiments of the present application are not limited to this. System applications refer to applications provided or developed by the manufacturer of the mobile terminal 100, and third-party applications refer to applications that are not provided or developed by the manufacturer of the mobile terminal 100. The manufacturer of the mobile terminal 100 may include the manufacturer, supplier, provider or operator of the mobile terminal 100. The manufacturer may refer to a manufacturer that manufactures electronic equipment with self-made or purchased parts and raw materials. The supplier may refer to a manufacturer that provides the entire machine, raw materials or parts of the electronic device. The operator may refer to a manufacturer responsible for the distribution of the electronic device.

In addition, in some embodiments of the present application, "smart home" is just a name that may be used by the application. In some other embodiments of the present application, the "smart home" application may also be called other terms such as "smart home", "intelligent home", "smart life", etc., and the name does not constitute any limitation to the present embodiment. The functions that can be achieved by "smart home" have been recorded in the method embodiments below and will not be repeated here.

The home device 200 refers to a home appliance product formed by introducing a microprocessor, sensor technology, and network communication technology into the home device. Therefore, when the home device is connected to the Internet, it also has the attributes of a social network. Therefore, the home device can also be understood as an IoT home appliance. In the embodiment of the present application, the home device 200 may specifically include: air conditioners, televisions, projection equipment, smart speakers, smart door locks, smart lights, smart windows, smart curtains, smart body fat scales, treadmills, smart rice cookers, smart electric kettles, and the like.

In the embodiment of the present application, the above-mentioned home device 200 has an automatic sensing function. The objects sensed by the home device 200 include not only sensing time and temperature, but also sensing the user's emotions, actions, behavioral habits, etc. Then, the state of the residential space and the state of the home appliance itself, the state of the home appliance service, the state of family members, etc. can be obtained. After that, some intelligent operations can be performed according to the acquired state of the residential space and the state of the home appliance itself, the state of the home appliance service, the state of family members, etc., that is, the home device can be automatically controlled; at the same time, as a component of the smart home, the home device 200 can be interconnected with the mobile terminal 100, as well as other home appliances and home furnishings and facilities in the home to form a system, so the home device 200 can receive the control instructions of the home user in the home or remotely through the mobile terminal, and perform linkage control on all home devices 200 in the home. It is worth noting that in addition to automatically setting and controlling the home device 200 according to the state of the residential space and the state of the user, the user can make personalized settings according to his own habits.

In some embodiments of the present application, the home device 200 may execute a corresponding event according to a control instruction sent by the mobile terminal 100 or according to a control instruction sent by the mobile terminal 100 through the server 300. Please refer to the detailed description below, and the embodiments of the present application will not be repeated here.

In other embodiments of the present application, the home device 200 can send the relevant data of the user performing the unlocking task to the mobile terminal 100, so that the mobile terminal 100 can determine whether the user has completed the unlocking task. In the process of the user using the home device 200 to perform the unlocking task, the home device 200 can perform continuous identity authentication on the user. Please refer to the detailed description below, and the embodiments of the present application will not be repeated here.

In other embodiments of the present application, the home device 200 can detect the user's status and send the detected user data to the mobile terminal 100 so that the mobile terminal 100 can determine the user's status. Please refer to the detailed description below, and the embodiments of the present application will not be repeated here.

The server 300 is a computer that runs faster and has a higher load than an ordinary computer. The server 300 has high-speed CPU computing power, long-term reliable operation, powerful I/O external data throughput capacity and better scalability. The server 300 can provide computing or application services for the mobile terminal 100, the home device 200, etc. in the network. In the embodiment of the present application, the server can be, but is not limited to, a network server, a database server, a cloud server, etc.

2A-2B, FIG. 3, and FIG. 4 respectively illustrate the software and hardware architecture of each device and server involved in the communication system 10.

Referring to FIG. 2A , FIG. 2A exemplarily shows a hardware structure of the mobile terminal 100 .

The mobile terminal 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the mobile terminal 100. In other embodiments of the present application, the mobile terminal 100 may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and/or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

The controller may be the nerve center and command center of the mobile terminal 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver/transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input/output (GPIO) interface, a subscriber identity module (SIM) interface, and/or a universal serial bus (USB) interface, etc.

In some embodiments of the present application, the processor 110 can detect the user identity and determine the control authority of the home device corresponding to the user according to the user identity. If the user has the authority to control a certain home device, the processor 110 can call the mobile communication module 150 or the wireless communication module 160 to send a control instruction to the corresponding home device through the server 300, or the processor 110 can call the wireless communication module 160 to directly send a control instruction to the corresponding home device. In addition, the processor 110 can also call the display screen 194 to display the status information of the home device. For details, please refer to the relevant description of the method embodiment and UI embodiment below, which will not be repeated here.

In other embodiments of the present application, the processor 110 detects whether the user breaks the usage rules according to the set usage rules of the mobile terminal 100. If the user breaks the usage rules, the processor can control the mobile terminal 100 to lock, or control the mobile terminal 100 to lock some functions. In addition, the processor 110 can also call the display screen 194 to display a prompt message for unlocking the mobile terminal 100. For details, please refer to the relevant description of the method embodiment and UI embodiment below, which will not be repeated here.

In other embodiments of the present application, the processor 110 may call the "smart home" application to obtain the user's schedule-related data recorded by the application program, and send the schedule-related data to the server 300. The server 300 is used to determine the user's schedule according to the obtained schedule-related data, and set the control rules of one or more home devices according to the user's schedule. Alternatively, the processor 110 may determine the user's schedule on its own end according to the obtained schedule-related data, and set the control rules of one or more home devices according to the user's schedule.

The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple groups of I2C buses. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces. For example: the processor 110 may be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 communicates with the touch sensor 180K through the I2C bus interface, thereby realizing the touch function of the mobile terminal 100.

The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to achieve communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 via the I2S interface to achieve the function of answering a call through a Bluetooth headset.

The PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface to implement the function of answering calls via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

The UART interface is a universal serial data bus for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is generally used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to implement the Bluetooth function. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 through the UART interface to implement the function of playing music through a Bluetooth headset.

The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the shooting function of the mobile terminal 100. The processor 110 and the display screen 194 communicate via the DSI interface to implement the display function of the mobile terminal 100.

The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 with the camera 193, the display 194, the wireless communication module 160, the audio module 170, the sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

The USB interface 130 is an interface that complies with the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the mobile terminal 100, and can also be used to transfer data between the mobile terminal 100 and peripheral devices. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other electronic devices, such as AR devices, etc.

It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the mobile terminal 100. In other embodiments of the present application, the mobile terminal 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the mobile terminal 100. While the charging management module 140 is charging the battery 142, it may also power the mobile terminal 100 through the power management module 141.

The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and/or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

The wireless communication function of the mobile terminal 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the mobile terminal 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

The mobile communication module 150 can provide solutions for wireless communications including 2G/3G/4G/5G applied to the mobile terminal 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the mobile terminal 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, demodulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

In some embodiments, the antenna 1 of the mobile terminal 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the mobile terminal 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and/or IR technology, etc. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and/or a satellite based augmentation system (SBAS).

In some embodiments of the present application, the mobile terminal 100 can establish a communication connection with the server 300 through the mobile communication module 150 and the wireless communication module 160, and then request a service from the server 300, and the server can control the home device 200 in response to the request. In other embodiments of the present application, the mobile terminal 100 can establish a communication connection with the home device 200 through the wireless communication module 160, and control the home device 200 based on the communication connection.

The mobile terminal 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the mobile terminal 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

In an embodiment of the present application, the display screen 194 can display the service status of the home device 200, information about family members, residential space status, etc. provided by the "smart home" application. For details, please refer to the relevant description of the method embodiment and UI embodiment below.

In some embodiments of the present application, a fingerprint sensor 180H, also known as an under-screen fingerprint, is also installed in the display screen 194. Specifically, when the electronic device turns on fingerprint recognition, the fingerprint collection area in the display screen 194 will emit light to penetrate the display panel, illuminate the user's fingerprint texture, and the fingerprint reflects the light to penetrate the panel and return to the image sensor, finally forming a fingerprint image for recognition.

In other embodiments of the present application, the display screen 194 may be used to display prompt information to prompt the user that the identity authentication is successful, the identity authentication fails, etc. Optionally, after the electronic device prompts the user that the identity authentication fails, it may also prompt the user to use the next authentication method for identity authentication. For details about the prompt information displayed on the display screen 194, please refer to the relevant description of the UI embodiment below, which will not be repeated here.

The mobile terminal 100 can realize the shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 194 and the application processor.

ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the mobile terminal 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

In some embodiments of the present application, the camera 193 may be used to capture a user's facial image for the mobile terminal 100 to perform facial recognition.

In some embodiments of the present application, the camera 193 may be used to capture images of the user's environment so that the mobile terminal 100 can determine the user's status, such as driving, exercising, doing housework, etc.

In an embodiment of the present application, the camera 193 can be used to capture a QR code image, and then the "smart home" application of the mobile terminal 100 can identify the information corresponding to the QR code, such as the logo and name of the public place to which the QR code belongs.

The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the mobile terminal 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

Video codecs are used to compress or decompress digital videos. The mobile terminal 100 may support one or more video codecs. Thus, the mobile terminal 100 may play or record videos in various coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of the mobile terminal 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

The internal memory 121 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM). In the embodiment of the present application, the NVM may store information about the area that the user has visited and the time spent in the area.

Random access memory may include static random-access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM, for example, the fifth generation DDR SDRAM is generally referred to as DDR5 SDRAM), etc.;

Non-volatile memory may include disk storage devices and flash memory.

Flash memory can be divided into NOR FLASH, NAND FLASH, 3D NAND FLASH, etc. according to the operating principle; single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), quad-level cell (QLC), etc. according to the storage unit potential level; universal flash storage (UFS), embedded multi media card (eMMC), etc. according to the storage specification.

The random access memory can be directly read and written by the processor 110, and can be used to store executable programs (such as machine instructions) of the operating system or other running programs, and can also be used to store user and application data, etc.

The non-volatile memory may also store executable programs and user and application data, etc., and may be loaded into the random access memory in advance for direct reading and writing by the processor 110 .

The external memory interface 120 can be used to connect to an external non-volatile memory to expand the storage capacity of the mobile terminal 100. The external non-volatile memory communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, files such as music and videos are stored in the external non-volatile memory.

In some embodiments of the present application, the above-mentioned memory is also used to store information of home devices, such as communication identification of home devices; user identity information, such as the user's "smart home" login account and password, the user's facial information, fingerprint information, voiceprint information, iris information, etc.; and associated storage of user identity information and permission information, for example: user A has permissions for all home devices, and user B has permissions for devices in the bedroom B and study at home.

In other embodiments of the present application, the memory is further used to store usage rules of the mobile terminal 100, which can be set by default by the mobile terminal 100 or can be customized by the user. The usage rules can, for example, specify the total time of using the mobile terminal 100 per day, the number of times it is used per day, and the time of each use, or specify the total time of using a certain application or a certain type of application installed in the mobile terminal 100 per day, the number of times it is used per day, and the time of each use, etc.

In other embodiments of the present application, the above-mentioned memory is also used to store user schedule-related data, and the user schedule-related data includes at least one of the following: social records, text messages, call records, emails, electronic notes, memos, etc.

The mobile terminal 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor.

The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

The speaker 170A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The mobile terminal 100 can listen to music or listen to a hands-free call through the speaker 170A.

The receiver 170B, also called a "earpiece", is used to convert audio electrical signals into sound signals. When the mobile terminal 100 receives a call or voice message, the voice can be received by placing the receiver 170B close to the ear.

Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting his mouth close to the microphone 170C to input the sound signal into the microphone 170C. The mobile terminal 100 can be provided with at least one microphone 170C. In other embodiments, the mobile terminal 100 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the mobile terminal 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the sound source, realize directional recording function, etc.

The earphone interface 170D is used to connect a wired earphone and can be a USB interface 130 or a 3.5 mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

In some embodiments of the present application, the mobile terminal 100 can provide voiceprint authentication for the user through the above-mentioned audio module 170, for example, prompting the user to input a sound signal through the speaker 170A/receiver 170B, and then collecting the sound signal input by the user through the microphone 170C, for the processor to match the sound signal with a pre-stored sound signal, and then determine whether the voiceprint authentication is passed.

In other embodiments of the present application, the audio module 170 is used to output voice information about the operating status of home devices bound to the "smart home".

In other embodiments of the present application, the audio module 170 is also used to output a prompt message of breaking the usage rules of the mobile terminal 100, and to output a voice prompt message of a to-do list when the user leaves home, etc.

The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be set on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor can be a parallel plate including at least two conductive materials. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The mobile terminal 100 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the mobile terminal 100 detects the touch operation intensity according to the pressure sensor 180A. The mobile terminal 100 can also calculate the touch position according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, an instruction to create a new short message is executed.

In the embodiment of the present application, the pressure sensor 180A can be used to collect pressure data applied by the user on a button or a display screen, and the mobile terminal 100 can perform identity authentication based on the user's previous operating habits.

The gyroscope sensor 180B can be used to determine the motion posture of the mobile terminal 100. In some embodiments, the angular velocity of the mobile terminal 100 around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor 180B. The gyroscope sensor 180B can be used for anti-shake shooting. Exemplarily, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the shaking of the mobile terminal 100, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the mobile terminal 100 through reverse movement to achieve anti-shake. The gyroscope sensor 180B can also be used for navigation and somatosensory game scenes.

The air pressure sensor 180C is used to measure air pressure. In some embodiments, the mobile terminal 100 calculates the altitude through the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.

The magnetic sensor 180D includes a Hall sensor. The mobile terminal 100 can use the magnetic sensor 180D to detect the opening and closing of the flip leather case. In some embodiments, when the mobile terminal 100 is a flip phone, the mobile terminal 100 can detect the opening and closing of the flip cover according to the magnetic sensor 180D. Then, according to the detected opening and closing state of the leather case or the opening and closing state of the flip cover, the flip cover automatic unlocking and other features are set.

The acceleration sensor 180E can detect the magnitude of the acceleration of the mobile terminal 100 in various directions (generally three axes). When the mobile terminal 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device and is applied to applications such as horizontal and vertical screen switching and pedometers.

The distance sensor 180F is used to measure the distance. The mobile terminal 100 can measure the distance by infrared or laser. In some embodiments, when shooting a scene, the mobile terminal 100 can use the distance sensor 180F to measure the distance to achieve fast focusing.

The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The mobile terminal 100 emits infrared light outward through the light emitting diode. The mobile terminal 100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the mobile terminal 100. When insufficient reflected light is detected, the mobile terminal 100 can determine that there is no object near the mobile terminal 100. The mobile terminal 100 can use the proximity light sensor 180G to detect that the user holds the mobile terminal 100 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode and pocket mode to automatically unlock and lock the screen.

The fingerprint sensor 180H is used to collect fingerprints. The mobile terminal 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc.

The temperature sensor 180J is used to detect temperature. In some embodiments, the mobile terminal 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the mobile terminal 100 reduces the performance of the processor located near the temperature sensor 180J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the mobile terminal 100 heats the battery 142 to avoid abnormal shutdown of the mobile terminal 100 due to low temperature. In other embodiments, when the temperature is lower than another threshold, the mobile terminal 100 boosts the output voltage of the battery 142 to avoid abnormal shutdown due to low temperature.

In the embodiment of the present application, the temperature sensor 180J and the humidity sensor can also detect relevant data of the ambient temperature and ambient humidity, so that the mobile terminal 100 can automatically control home appliances according to the current ambient temperature.

The touch sensor 180K is also called a "touch panel". The touch sensor 180K can be set on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be set on the surface of the mobile terminal 100, which is different from the position of the display screen 194.

The ambient light sensor 180L is used to sense the ambient light brightness. The mobile terminal 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the mobile terminal 100 is in a pocket to prevent accidental touches.

In an embodiment of the present application, when the mobile terminal 100 passes a password authentication, such as a character password authentication or a pattern password authentication, the touch sensor 180K can be used to collect the user's touch operation on the display screen, determine the character password or pattern password input by the user, etc.

The bone conduction sensor 180M can obtain a vibration signal. In some embodiments, the bone conduction sensor 180M can obtain a vibration signal of a vibrating bone block of the vocal part of the human body. The bone conduction sensor 180M can also contact the human pulse to receive a blood pressure beat signal. In some embodiments, the bone conduction sensor 180M can also be set in an earphone and combined into a bone conduction earphone. The audio module 170 can parse out a voice signal based on the vibration signal of the vibrating bone block of the vocal part obtained by the bone conduction sensor 180M to realize a voice function. The application processor can parse the heart rate information based on the blood pressure beat signal obtained by the bone conduction sensor 180M to realize a heart rate detection function.

The ECG sensor can be used to collect the user's ECG data and then determine the user's status, such as whether the user is in motion.

The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The mobile terminal 100 may receive key input and generate key signal input related to user settings and function control of the mobile terminal 100.

Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 194, motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

Indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

The SIM card interface 195 is used to connect the SIM card. The SIM card can be connected to and separated from the mobile terminal 100 by inserting it into the SIM card interface 195 or pulling it out from the SIM card interface 195. The mobile terminal 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, etc. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The mobile terminal 100 interacts with the network through the SIM card to implement functions such as calls and data communications. In some embodiments, the mobile terminal 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the mobile terminal 100 and cannot be separated from the mobile terminal 100.

The software system of the mobile terminal 100 may adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. The present application embodiment takes a mobile operating system of a layered architecture as an example to exemplify the software structure of the mobile terminal 100.

Referring to FIG. 2B , FIG. 2B exemplarily shows a software structure block diagram of the mobile terminal 100 according to an embodiment of the present application.

The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the mobile operating system is divided into four layers, from top to bottom: application layer, program framework layer/core service layer, underlying library and runtime, and kernel layer.

The application layer can include a series of application packages.

As shown in FIG. 2B , the application package may include “smart home” applications, camera, memo, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and other applications.

In the embodiment of the present application, the "Smart Home" application can bind the family members to form a family group, and can also bind home devices, including mobile terminals of family members and home devices, etc. Users can manage and control home devices through the "Smart Home" application, or the "Smart Home" application can automatically control the bound home devices and display corresponding prompt information.

The program framework layer provides application programming interface (API) and programming framework for the application programs in the application layer. The program framework layer includes some predefined functions.

As shown in FIG. 2B , the program framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

The phone manager is used to provide communication functions of the mobile terminal 100, such as management of call status (including connected, disconnected, etc.).

The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.

The runtime can refer to all code libraries, frameworks, etc. required for the program to run. For example, for the C language, the runtime includes a series of function libraries required for the C program to run. For the Java language, in addition to the core library, the runtime also includes the virtual machine required for the Java program to run. The above core library may include the function functions that the Java language needs to call.

The underlying library may include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

A 2D graphics engine is a drawing engine for 2D drawings.

The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

The following is an illustrative description of the workflow of the software and hardware of the mobile terminal 100 in conjunction with the capture and photo shooting scene.

When the touch sensor 180K receives a touch operation, the corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes the touch operation into a raw input event (including touch coordinates, timestamp of the touch operation, and other information). The raw input event is stored in the kernel layer. The application framework layer obtains the raw input event from the kernel layer and identifies the control corresponding to the input event. For example, if the touch operation is a touch single-click operation and the control corresponding to the single-click operation is the control of the camera application icon, the camera application calls the interface of the application framework layer to start the camera application, and then starts the camera driver by calling the kernel layer to capture static images or videos through the camera 193.

In the embodiment of the present application, one or more electronic devices involved in the communication system 10, such as electronic device 101, electronic device 102 and electronic device 103, have the same hardware and software architecture as the mobile terminal 100. Please refer to the detailed description of Figures 2A-2B above, and the embodiment of the present application will not go into details.

FIG3 is a block diagram of the hardware structure of the home device 200 provided in an embodiment of the present application.

As shown in FIG3 , the home device 200 may include: one or more processors 201, a memory 202, a communication interface 203, a transmitter 205, a receiver 206, a coupler 207, and an antenna 208. These components may be connected via a bus 204 or other means, and FIG3 takes the connection via a bus as an example. Among them:

The processor 201 can be used to read and execute computer-readable instructions. Specifically, the processor 201 can be used to call a program stored in the memory 202, such as a method for controlling the home device 200 to perform a certain operation provided in an embodiment of the present application, an implementation program on the home device 200 side, and execute instructions contained in the program.

The memory 202 is coupled to the processor 201 and is used to store various software programs and/or multiple sets of instructions. Specifically, the memory 202 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more disk storage devices, flash memory devices or other non-volatile solid-state storage devices.

The memory 202 can store an operating system (hereinafter referred to as system), such as embedded operating systems such as uCOS, VxWorks, RTLinux, etc. The memory 202 can also store a network communication program, which can be used to communicate with the server 300, the mobile terminal 100, etc.

In the embodiment of the present application, the memory 202 of the home device 200 can store user information, such as the user's operating habits, the user's facial information, fingerprint information, voiceprint information, etc. The information of the home device includes the communication identifier of the home device, etc. Then the server 300 can send information to it according to the communication identifier of the home device. For example, send a control instruction to the home device, feedback the status of the home device to the mobile terminal, etc.

The communication interface 203 can be used for the server 300 to communicate with other communication devices, such as the mobile terminal 100, the home device 200, etc. Specifically, the communication interface 203 can be a 3G communication interface, a long-term evolution (LTE) (4G) communication interface, a 5G communication interface, a WLAN communication interface, a WAN communication interface, etc. Not limited to a wireless communication interface, the server 300 can also be configured with a wired communication interface 203 to support wired communication.

In some embodiments of the present application, the transmitter 205 and the receiver 206 can be regarded as a wireless modem. The transmitter 205 can be used to transmit and process the signal output by the processor 201. The receiver 206 can be used to receive the signal. In the server 300, the number of the transmitter 205 and the receiver 206 can be one or more. The antenna 208 can be used to convert the electromagnetic energy in the transmission line into electromagnetic waves in the free space, or to convert the electromagnetic waves in the free space into electromagnetic energy in the transmission line. The coupler 207 can be used to divide the mobile communication signal into multiple paths and distribute it to multiple receivers 206. It can be understood that the antenna 208 of the network device can be implemented as a large-scale antenna array.

It should be noted that the server 300 shown in FIG. 3 is only one implementation of the embodiment of the present application. In actual applications, the server 300 may also include more or fewer components, which is not limited here.

FIG4 is a block diagram of the hardware structure of the server 300 provided in an embodiment of the present application.

As shown in FIG4 , the server 300 may include: one or more processors 301, a memory 302, a communication interface 303, a transmitter 305, a receiver 306, a coupler 307, and an antenna 308. These components may be connected via a bus 304 or other means, and FIG4 takes the bus connection as an example. Among them:

The processor 301 can be used to read and execute computer-readable instructions. Specifically, the processor 301 can be used to call a program stored in the memory 302, such as a method for controlling the home device 200 to perform a certain operation provided in an embodiment of the present application, an implementation program on the home device 200 side, and execute instructions contained in the program.

The memory 302 is coupled to the processor 301 and is used to store various software programs and/or multiple sets of instructions. Specifically, the memory 302 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more disk storage devices, flash memory devices or other non-volatile solid-state storage devices.

The memory 302 can store an operating system (hereinafter referred to as system), such as embedded operating systems such as uCOS, VxWorks, RTLinux, etc. The memory 302 can also store a network communication program, which can be used to communicate with the server 300, the mobile terminal 100, etc.

In some embodiments of the present application, the memory 302 of the server 300 may store the registration information of the user registering the "smart home" application. The user's registration information includes the user account, password, and the user's biometrics, such as fingerprint, face, voiceprint information, etc., as well as the user's private space in the home, etc. When the user uses the user account to log in to the "smart home" application in the mobile terminal 100, the server 300 can establish a communication connection with the mobile terminal 100 through the communication interface 303, and provide application services to the mobile terminal 100 based on the communication connection. The memory 302 of the server 300 can store information about home devices 200 in the home, and the information of the home devices 200 includes: the communication identification of the home device, the location information of the home device in the home space, and so on.

In other embodiments of the present application, the memory 302 of the server 300 may store usage rules of the home device, and the usage rules may be the default settings of the server 300, or the default settings of the mobile terminal and uploaded to the server 300, or may be user settings, and the specific user may directly write the usage rules into the server 300, or the user may set the usage rules in the mobile terminal 100, and the mobile terminal 100 may upload the usage rules to the server 300. The embodiments of the present application are not limited to this.

The constraint objects of the above-mentioned usage rules may be any one or more household devices, which may be mobile terminals 100 and/or household devices 200. Specifically, the above-mentioned usage rules may be, for example, restrictions on the total daily usage time, daily usage times, each usage time, usage functions, usage APPs, etc. of the mobile terminal 100-1. Usage functions may be, for example, payment functions, functions for deleting text messages, phone calls and other social records, etc., and usage APPs may be, for example, live broadcast APPs, game APPs, online gambling APPs, etc. Alternatively, it is also possible to set the time and number of times to use some functions in a certain APP or to use a certain APP. The embodiments of the present application do not limit this.

In other embodiments of the present application, the memory 302 of the server 300 may also store user schedule-related data sent by the user mobile terminal 100, and the server 300 is used to determine the user's schedule based on the acquired schedule-related data, and set control rules for one or more home devices based on the user's schedule.

The above-mentioned user schedule-related data includes any one or more of the following: social records, text messages, call records, emails, electronic notes, memos, etc.

The "Smart Home" application is an application for controlling home devices. To achieve the control function, the "Smart Home" application needs to obtain the following functions in the mobile terminal 100: phone, positioning, Bluetooth; and read the following data: user's personal information, such as Huawei account, login password, nickname, avatar, device and network information, such as device identifier, device name, model, version, manufacturer, MAC address, IP address, device operating status, custom attributes, setting parameters, device events and usage records, used to discover and bind devices, control, manage and upgrade on the go; it will also use the data required for device services, including: device detection and recorded data, used to view the device's working status, detection data, statistical reports and historical records; data required for problem location such as application and device log information.

The communication interface 303 can be used for the server 300 to communicate with other communication devices, such as the mobile terminal 100, the home device 200, etc. Specifically, the communication interface 303 can be a 3G communication interface, a long-term evolution (LTE) (4G) communication interface, a 5G communication interface, a WLAN communication interface, a WAN communication interface, etc. Not limited to a wireless communication interface, the server 300 can also be configured with a wired communication interface 303 to support wired communication.

In an embodiment of the present application, the server 300 can receive, through the communication interface 303, the registration information of the user registering the "smart home" application sent by the mobile terminal 100, as well as control instructions for home devices, and status information of the home devices, status information of the home environment, status information of family members, etc. fed back by the home devices.

In some embodiments of the present application, the transmitter 305 and the receiver 306 can be regarded as a wireless modem. The transmitter 305 can be used to transmit and process the signal output by the processor 301. The receiver 306 can be used to receive the signal. In the server 300, the number of the transmitter 305 and the receiver 306 can be one or more. The antenna 308 can be used to convert the electromagnetic energy in the transmission line into electromagnetic waves in the free space, or to convert the electromagnetic waves in the free space into electromagnetic energy in the transmission line. The coupler 307 can be used to divide the mobile communication signal into multiple paths and distribute it to multiple receivers 306. It can be understood that the antenna 308 of the network device can be implemented as a large-scale antenna array.

It should be noted that the server 300 shown in FIG. 4 is only one implementation of the embodiment of the present application. In actual applications, the server 300 may also include more or fewer components, which is not limited here.

Next, a method for controlling a home device provided by the present application is introduced in conjunction with a UI embodiment.

5A-5D exemplarily show schematic diagrams for introducing concepts related to “family”.

Family members are a name for the different relationships between users under the concept of home. In today's society, there is an organizational and division of labor relationship between users in a family. In other words, in a family, it is necessary to distinguish the roles of each member. For example, they can be divided into four roles: family creator, administrator, ordinary user, designated user, and family visitor. These four member roles have different levels of control rights for home devices. Among them,

The family owner is also called the creator: equivalent to the head of the family, who can create a family. The family owner has the highest authority, that is, the creator's authority level is the first level authority level. The creator can manage all family members and control all family devices, including adding and deleting family members, setting the roles of family members, and the device control permissions they have, etc.

Administrator: The administrator has the second highest authority, that is, the administrator's authority level is the second level authority level. The administrator is responsible for managing all of the family, such as adding, deleting, and modifying rooms, members, and devices. In addition, a family may have multiple administrators, for example, the father, mother, or grandparents of the family can all be administrators of the family.

Ordinary users: Ordinary users can only use home devices, and their permission level for using home devices is usually lower than that of the family administrator.

Designated users: For example, elderly people, children and other users with limited behavioral abilities. Designated users, like ordinary users, only have the right to use home devices, but the scope of their permissions may be different. For example, when designated users are elderly people, children and other users with limited behavioral abilities, their scope of permissions does not include kitchen appliances with a high risk index.

Home visitors: Home visitors can also be called temporary users, which means they are not permanent users of the family, such as housekeepers, tutors, etc. Such users will only appear in the family during a certain period of time and use the devices in the family. When home visitors are not in the family space, they do not have the authority to control home devices.

It is understandable that the above-mentioned designated users, temporary users, etc. are not set in every smart home, and the role settings of family members can be set and changed by the family owner according to the actual situation of the family. The embodiment of the present application does not limit this.

Referring to FIG. 5A , FIG. 5A exemplarily shows the family members included in a family and the roles played by the family members in the family relationship.

As shown in Fig. 5A, the family members specifically include Xiaozhong, Xiaohua, Xiaoming, Xiaohong, Xiaoai, Xiaoyu and Xiaoxue, etc. Among them, Xiaozhong and Xiaohua are Xiaoai's father and mother respectively, and Xiaoming and Xiaohong are Xiaoai's grandparents respectively, Xiaoyu is a visitor of the family, Xiaoyu can be Xiaoai's tutor, or Xiaohua's personal coach, etc., and Xiaoxue may be a housekeeper, usually taking care of the young Xiaoai, or the daily life of the elderly Xiaoming or Xiaohong, and the hygiene of the family, etc. In the embodiment of the present application, each of the above family members can be equipped with one or more mobile terminals, such as mobile phones, tablets, notebooks, smart watches, etc. Specifically including Xiaozhong's mobile terminal 100-1, Xiaohua's mobile terminal 100-2, Xiaoming's mobile terminal 100-3, Xiaohong's mobile terminal 100-4, Xiaoai's mobile terminal 100-5 and mobile terminal 100-6, Xiaoyu's mobile terminal 100-7 and Xiaoxue's mobile terminal 100-8, etc. These terminal devices all belong to the small and medium-sized smart home communication system, and each terminal device can be interconnected.

Referring to FIG. 5B , FIG. 5B exemplarily shows the authority levels respectively possessed by the family members of Xiaozhong.

As shown in FIG5B , the creator of Xiaozhong's home is Xiaozhong himself, and the permission level of the creator is level 1. Level 1 permission can manage all family members and control all home devices, including adding and deleting family members, setting the roles played by family members, and the device control permissions they have.

Among them, Xiaohua and Xiaohong are administrators, and the administrator's authority level is level 2. The scope of level 2 authority includes managing all of the family, such as adding, deleting, and modifying rooms, members, and devices, etc. In addition, they can only control the use of devices in the rooms they own.

Among them, Xiao Ming and Xiao Ai are designated users, and their permission level is level 3. Level 3 permission only includes the permission to control the use of home devices, but the control scope is only the devices in the room they own, except for kitchen appliances with a high risk index.

Among them, Xiaoyu and Xiaoxue are home visitors, and their permission level is level 4. The scope of level 4 permission includes that only when the home visitor is in the home space, he has the permission to control the home devices.

The above-mentioned authority levels decrease in order from level one to level four, and the scope of authority also decreases in order.

In the embodiments of the present application, the division of roles and levels of family members are merely examples. For example, different designated users, namely the elderly and children, may have different levels of authority, and the embodiments of the present application do not impose any restrictions on this.

The above article briefly introduces the permission levels of different family members, including the management of rooms, family members, and home devices. Next, we will specifically introduce the permissions of family members to control home devices based on the rooms owned by family members, which will not be repeated here.

A family is not only a concept of a personal relationship between users, but also a mapping of a physical space. In the smart home scenario, the most common concept of space is a room. According to physical buildings, in addition to flat rooms, there are also three-dimensional floors. However, for the vast majority of family buildings in society, one-story family houses far outnumber villas or duplex houses. Therefore, the embodiments of this application are only introduced with a one-story family space.

Referring to FIG. 5C , FIG. 5C exemplarily shows the family floor plan structure.

As shown in Figure 5C, the layout structure of the family includes: bedroom A, bedroom B, bedroom C, living room, kitchen, study, gym, recreation room, etc. Among them, bedroom A is the private space belonging to Xiaozhong and Xiaohua, bedroom B is the private space belonging to Xiaoming and Xiaohua, bedroom C belongs to Xiaoai, and the living room, kitchen, study, recreation room, and gym are the public areas of family members. However, since Xiaoai is still a child, in order to protect Xiaoai's safety, Xiaoai is prohibited from operating the household equipment in the kitchen without adult supervision. In the embodiment of the present application, private space and public space are customized according to the actual situation of different families, that is, the bedroom is not necessarily a private space. For example, a family has a baby, and the baby room belongs to the baby's bedroom, but for the baby, she does not have the ability to act independently, and needs the care of other people in the family from time to time, which obviously does not exist in private space. On the contrary, for children like Xiaoai, they are eager to have their own private space. Therefore, the private space and public space defined in the embodiment of the present application are not absolute concepts and are not limited.

In addition to mapping with physical space, home space has another important function - device binding, which can also be understood as the spatial classification of devices. This is because, in a family, most devices, almost all home devices except mobile terminals, are fixed in a certain room. Therefore, when a user is looking for a device or controlling or using a device, as long as he knows the room to which the device belongs and remembers the room where the device is located, he can quickly find the device. In addition, although all members of a family have a very close relationship, everyone is an individual and needs his own private space. Therefore, a family is actually divided into multiple spaces, with their own private space and public space for everyone. This can not only protect personal privacy and security, but also enhance the relationship between family members.

Refer to Figure 5D, which exemplarily shows the home space layout structure and a schematic diagram of the home equipment installed in each space.

As shown in FIG5D , a home is equipped with multiple home devices 200, including: home device 200-1, home device 200-2, home device 200-3, home device 200-4, home device 200-5, home device 200-6, etc. The home devices may specifically be smart devices such as televisions, refrigerators, air conditioners, washing and drying machines, range hoods, kitchen appliances, air purifiers, fresh air blowers, humidifiers and dehumidifiers, sweepers, cameras, door locks, electric curtains, water dispensers, and lights.

In the embodiment of the present application, the above-mentioned mobile terminals, household devices, etc. are all household devices in the family, and an interconnected communication system can be obtained by networking all household devices, which is the communication system 10 shown in FIG. 1B above, and the above-mentioned series of mobile terminals are all mobile terminals 100 in the above-mentioned communication system 10, and the above-mentioned series of household devices are all household devices 200 in the above-mentioned communication system 10. In other embodiments of the present application, the communication system 10 can also be called a "smart home" system.

In the above communication system 10, users can download and install the "Smart Home" application in the mobile terminal, register and log in to the account in the "Smart Home", bind home devices, etc. After that, users can view, manage, and control home devices and electronic devices through the application. Among them, the definition of the "Smart Home" application can be referred to the previous introduction, which will not be repeated here. The following is an exemplary introduction to the functions and UI of the "Smart Home" application, as follows:

Figures 5A-5D only introduce the family scene, including the family space and the relationship between family members. The method provided in the embodiment of the present application can also be applied to the office scene, and the office space and company employees are not described one by one here.

6A-6V and 7A-7I exemplarily illustrate the operations related to the user creating a "smart home". The operations of creating a "smart home" may specifically include:

1. Register and log in to the "Smart Home" account to generate a virtual home space on the mobile terminal to facilitate centralized control and management of the family.

2. Add home devices, including binding the devices to the home and configuring the network for the home devices. Device configuration refers to the process of connecting the device to the Internet so that after the device is powered on, it can transmit data between the "Smart Home" platform. The home creator, that is, the home owner, can also manage the added home devices, including: deleting, replacing, classifying, etc.

3. Add family members, including adding administrators, ordinary users, family visitors, etc. The family owner (family) and family administrator can also manage family members and family devices, including adding, deleting, changing, and controlling family devices, as well as adding, deleting, and changing family members, etc. Ordinary users can only control and use family devices, and family visitors can only control family devices within their authority at specific times, such as when they are in the family space. First, Figure 6A takes Dad Xiaozhong's mobile terminal 100-1 as an example to introduce the user interface diagram provided by "Smart Home" after Xiaozhong logs in to the "Smart Home" account.

As shown in FIG. 6A , the user interface 61 displays: a home name option 611 , an add control 612 , a device navigation bar 613 , a device display area 614 , and a page navigation bar 615 .

Among them, the family name is "**"'s home, "**" is the account name of the current logged-in "Smart Home" user. At this time, the account logged in on the mobile terminal 100-1 is Xiaozhong's account, and the displayed content of the family name option 611 is Xiaozhong's home.

Among them, the adding control 612 is used to add home devices, that is, to bind the home devices to the small home and configure the network for the home devices.

Among them, the device navigation bar 613 includes: all options 613A and classification options 613B. When the all option 613A is selected, all the home devices that the user has added are displayed in the device display area 614. When the user has not added any device, the device display area 614 does not display any home device. When the classification option 613B is selected, the user can classify the home devices that have been added, such as dividing the home devices into spaces in the home, etc. The operation of classifying home devices can be referred to the detailed description below, which will not be repeated here.

Among them, the device display area 614 is used to display all or part of the household devices in Xiaozhong’s home.

The page navigation bar 615 includes: home page 615A, mall 615B, smart 615C, and my 615D. The home page 615A is used to display the user interface 610 introduced above. The mall 615B is used to jump to the shopping interface to purchase the equipment needed for the family, etc. For the sub-pages provided by smart 615C and my 615D, please refer to the detailed description of the UI embodiment below, which will not be repeated here.

Next, Figures 6B to 6K exemplarily show the relevant operation interfaces for a user to add home devices.

6B to 6E exemplarily illustrate the operation interface of the automatic adding device.

When mobile terminal 100 - 1 displays user interface 61 shown in FIG. 6A , an operation acting on add control 612 may be detected, and in response to the operation, mobile terminal 100 - 1 displays user interface 62 shown in FIG. 6B .

FIG. 6B exemplarily shows a schematic diagram of a user interface of a scanning device.

As shown in FIG6B , the user interface 62 shows a user interface schematic diagram of automatically scanning nearby devices. At this time, the mobile terminal 100-1 may automatically turn on near-field communication functions such as Bluetooth and WI-FI, send a detection signal to nearby devices, and search for nearby devices. When the nearby device receives the detection signal and replies a detection response to the mobile terminal, the mobile terminal 100-1 displays the user interface 62 shown in FIG6C .

FIG. 6C exemplarily shows a schematic diagram of a user interface for adding a device.

As shown in FIG6C , the user interface 62 displays a user interface diagram of the scanned nearby devices. For example, the attached mobile terminals include: smart door locks, televisions, cameras, electric curtains, etc. An add control is displayed for each scanned device. The user can click on the add control corresponding to any one or more devices to add the scanned device to the home device, that is, bind it to the home.

When the mobile terminal 100-1 displays the user interface 62 shown in FIG. 6C, an operation of adding a control corresponding to the smart door lock may be detected. In response to the operation, the mobile terminal 100-1 displays the user interface 62 shown in FIG. 6D.

As shown in FIG. 6D , the add control corresponding to the smart door lock changes from the original “Add” to “Added” for display.

When mobile terminal 100 - 1 displays user interface 62 shown in FIG. 6D , an operation acting on a return control may be detected, and in response to the operation, mobile terminal 100 - 1 displays user interface 61 shown in FIG. 6E .

As shown in FIG. 6E , at this time, the device display area 614 changes from having no device in the original device display area 614 shown in FIG. 6A to a household device, namely a smart door lock, that has been added to Xiaozhong's home.

The above-mentioned Figures 6A-6E merely illustrate a method of adding a device, that is, the mobile terminal 100-1 automatically scans nearby electronic devices. If the nearby electronic devices are powered on and any wireless communication function corresponding to the mobile terminal 100-1 is turned on, a detection response can be sent to the mobile terminal 100-1, so that the mobile terminal 100-1 can search for the device, add it to the home device, and pair it with the network. The above-mentioned wireless communication functions that are turned on include any one or more of the following: WLAN, BT, IR, ZigBee, etc. Therefore, the method of adding devices shown in Figures 6A-6E can also be called Bluetooth network pairing, WiFi network pairing, or other network pairing methods, and binding the device.

However, in the embodiment of the present application, there are multiple ways to add devices. For example, the user can add a home device by clicking the manual add control 621 shown in FIG. 6B .

6F to 6K exemplarily illustrate schematic diagrams of the operation interface for a user to manually add a device.

As shown in FIG6F, the mobile terminal 100-1 detects the operation of the user acting on the manual addition control 621, and displays the user interface 63 shown in FIG6G, which includes: a series of icons of home devices of different types and models stored in advance. As shown in FIG6G, the mobile terminal 100-1 can detect the operation of the user selecting the icon of the home device to be added, and in response to the operation, the mobile terminal 100-1 displays the network configuration user interface 64 shown in FIG6H. Afterwards, the mobile terminal 100-1 detects the operation of the user entering the home Wi-Fi password and clicking the next control as shown in FIG6I. In response to the operation, the mobile terminal 100-1 sends the home Wi-Fi account and password to the TV, so that the TV connects to the router through the home Wi-Fi account and password, and communicates with the server 300 in the smart home system. After the TV connects to the router through the home Wi-Fi account and password, and communicates with the server 300 in the smart home system, it can send a message of successful network configuration to the mobile terminal 100-1, and the mobile terminal 100-1 will display a prompt message of successful network configuration as shown in FIG6J. It is understandable that before the mobile terminal 100-1 enters the password of the home Wi-Fi and clicks next, it is necessary to ensure that the TV is powered on, and by long pressing the power button or reset button of the home device, the TV enters the sniffer mode, which can also be called the promiscuous mode. At this time, the device does not filter data packets and can monitor all 802.11 packets in the ring. At this time, the user can configure the network for the home device on the mobile terminal 100-1.

In addition to the above two methods of automatic addition and manual addition for Bluetooth and WiFi network configuration, you can also add devices through NFC network configuration, such as touching or scanning QR codes.

Specifically, the user can add the device by clicking the scan code add control 622 shown in Figure 6B and then scanning the QR code label attached to the home device. The embodiment of the present application will not be described in detail here.

In other embodiments of the present application, the mobile terminal 100-1 can also configure the network for the home device and bind it to the home in other ways, such as sound wave configuration, that is, the mobile phone sends sound waves to transmit the WiFi SSID, password and other information to the device, the device recognizes and completes the initialization process, and establishes a network connection with the smart home platform.

After that, every time a new smart power supply is added to the home, the user can configure the network for the new home device and add it to the home devices through the various methods described above.

As shown in FIG6K , FIG6K exemplarily shows a user interface diagram after Xiaozhong adds all the household devices in his home to the household devices. It can be seen that at this time, the device display area 614 in the home page displays: smart door locks, TV A, cameras, electric curtains A, body fat scales, desk lamps, electric kettles, rice cookers, refrigerators, TV B, air purifiers A, air conditioners A, smart lights A, TV B, electric windows B, and air conditioners B, etc., which are not listed one by one here. The user can browse all household devices by sliding the device display area 614 under the navigation.

Next, Figures 6L to 6V exemplarily show schematic diagrams of the operation interface for a user to classify home devices.

When the mobile terminal 100-1 detects that the user clicks the classification option 613B shown in FIG6K, the mobile terminal 100-1 switches the device display area 614 from all the home devices shown in FIG6K to some of the classified home devices shown in FIG6L.

As shown in FIG6L, when the category is selected, the device display area 614 does not display any icons of home devices, because the home devices have not been classified in the small. At this time, the mobile terminal 100-1 can detect the user's operation of clicking the category management control 616 shown in FIG6L, and in response to the operation, the mobile terminal 100-1 displays the category management user interface 65 shown in FIG6M.

As shown in FIG6M , the category management user interface 65 displays a sorting control 651A, an add room control 651B, and a delete control 651C. The sorting control 651A is used to sort the display order of the added room types, the add room control 651B is used to add a room type, and the delete control 651C is used to delete the added room type. At this time, the mobile terminal 100-1 can detect the user's operation of clicking the add room control 651B shown in FIG6N , and in response to the operation, the mobile terminal 100-1 displays the category management user interface 65 shown in FIG6N .

As shown in FIG. 6N , the management user interface 65 displays a window 652 for adding a room. The user can enter a custom room name such as “Bedroom A” and then click a confirmation control. In response to the user operation, the mobile terminal 100 - 1 displays the user interface 65 as shown in FIG. 6O .

As shown in FIG6O , the management user interface 65 displays a list item 653A of bedroom A, and the list item 653A displays detailed device information of bedroom A, such as 0 devices, to prompt the user that no home devices have been added to bedroom A. At this time, the mobile terminal 100-1 can detect that the user clicks on the list item 653A of FIG6O , and in response to the operation, the mobile terminal 100-1 displays the detailed interface 66 of bedroom A shown in FIG6P .

As shown in Fig. 6P, the details interface 66 of bedroom A displays an add device control 654A and a delete device control 654B. At this time, the mobile terminal 100-1 can detect the user's click on the add device control 654A of Fig. 6P, and in response to this operation, the mobile terminal 100-1 will pop up the add device window 655 shown in Fig. 6Q.

As shown in Fig. 6Q, the details interface 66 of bedroom A displays an add device window 655. The add device window 655 displays the icons of all the home devices that the user has added above.

As shown in FIG6R , the user can select the devices belonging to the bedroom A, such as the TV A and the air purifier A, and then click the confirmation control. In response to the above operation, the mobile terminal 100-1 will display the details interface 66 of the bedroom A shown in FIG6S . At this time, the mobile terminal 100-1 can detect the operation of the user clicking the return control shown in FIG6S . In response to this operation, the mobile terminal 100-1 will display the classification management interface 65 shown in FIG6T .

As shown in FIG. 6T , the list item 653A in the classification management interface 65 changes from the original prompt information showing 0 devices as shown in FIG. 6O to the prompt information showing 2 devices.

Based on the classification operation of home devices introduced in 6K-6T above, users can repeat the above operations to add more rooms and add devices belonging to each room.

Referring to FIG. 6U , FIG. 6U exemplarily shows a classification result displayed by the mobile terminal 100 - 1 .

As shown in FIG6U , the user interface 65 displays list items 653A, 653B, 653C, 653D, 653E, 653F (not shown), 653G (not shown), and 653H (not shown), etc. The user can browse the unshown list items by sliding the user interface 65 up and down, and the rooms corresponding to the above list items are bedroom A, bedroom B, bedroom C, living room, kitchen, study, gym, entertainment room, etc. At this time, the mobile terminal 100-1 can detect the operation of the user clicking the return control shown in FIG6U , and in response to the operation, the mobile terminal 100-1 displays the user interface 61 shown in FIG6V .

As shown in FIG6V , the device navigation bar 613 of the user interface 61 includes: all options 613A, bedroom A options 613C, bedroom B options 613D, bedroom C options 613E, living room options 613F, kitchen options 613G (not shown), study room options H (not shown), gym options I (not shown), entertainment room options J, etc. (not shown). The user can browse the options not shown by sliding the device navigation bar 613 area left and right.

As shown in FIG6V , at this time, the device navigation bar 613 selects the bedroom A option 613C. The device display area 614 displays the icons of the household devices contained in the bedroom A. The user can also click other options in the device navigation bar 613 to make the device display area 614 display the icons of the devices contained in the room corresponding to the other options. In the embodiment of the present application, the device display area 614 not only displays the icon of the device, but also displays the working status of the device corresponding to the icon, such as whether the device is turned on or off.

It is worth noting that the user can click the icon of the device in the device display area 614 to enter the device details page to control the device. Please refer to the detailed introduction below for details.

In the embodiment of the present application, the series of operations of the user manually dividing the room and the equipment in the room shown in the above FIGS. 6K to 6U can also be referred to as the second operation.

It is understandable that, in addition to the user-initiated classification shown in the above-mentioned Figures 6L to 6V, the classification of the rooms of the home devices can also be automatically classified by the mobile terminal 100-1. The details are as follows:

Some of the home devices bound to the Smart Home APP include sweeping robots, vacuum cleaners, and other devices that can move around the room. When these devices are turned on, they can obtain the room layout information of the entire home, that is, the number of rooms in the home, the location of the rooms, the area of the rooms, etc., as well as the information of the home appliances configured in the room.

Taking a sweeping robot as an example, it has numerous sensors, such as cameras, laser ranging sensors, wall sensors, collision sensors, ultrasonic radar sensors, electronic compass sensors, cliff sensors, gyroscopes/accelerometers, fall sensors, odometer sensors, and so on.

First, the robot vacuum cleaner can use its various sensors to detect the surrounding environment, and use the perceived room environment information to construct a map of the entire home, and then obtain the room layout information of the home. The room layout information includes: the number of rooms, the location of the rooms, the area of the rooms, etc.

In addition, the sweeping robot can also scan household appliances configured in each room. For example, one room is equipped with kitchen appliances such as rice cookers and electric kettles; another room is equipped with a smart toilet; another room is equipped with a bed and a wardrobe; another room is equipped with bookcases, desks, computers, etc.

Afterwards, the sweeping robot uploads the room information detected by the sensor and the equipment information in each room to the processor, and the processor matches these data, that is, classifies the room type according to the type of household equipment configured in each room. For example, a room with kitchen appliances such as rice cookers and electric kettles is classified as a kitchen; a room with a smart toilet is classified as a bathroom; a room with a bed and a wardrobe is classified as a bedroom; a room with a bookcase, a desk, and a computer is classified as a study, etc. In the embodiment of the present application, it is assumed that the room layout and equipment layout of the small family generated by the sweeping robot can specifically be the family structure exemplified in Figure 5D above, which will not be described one by one here.

Finally, the sweeping robot can send the above division results to the mobile terminal 100 through the server 300, or directly send them to the mobile terminal 100, which is not limited in the embodiment of the present application.

In some embodiments of the present application, the sweeping robot may also send the raw data of the room layout information and the corresponding device layout information detected by the sensor to the mobile terminal 100, and the processor of the mobile terminal 100 may classify the room type according to the type of equipment configured in each room. In other embodiments of the present application, the sweeping robot may also send the raw data of the room layout information and the corresponding device layout information detected by the sensor to the server 300, and the server 300 may classify the room type according to the type of equipment configured in each room, and then send the classification result to the mobile terminal 100 and the sweeping robot and other devices, and the embodiments of the present application are not limited to this.

In some embodiments of the present application, when the above-mentioned method is used in the mobile terminal 100 to automatically divide rooms and devices, the division result may be inaccurate; or after the rooms and devices are automatically divided, the location of some devices is moved, resulting in a change in the room to which the device belongs; or the family remodels the house, for example, the original study is arranged as a bedroom, etc. At this time, the mobile terminal 100 can support the user to manually reset the room information and device information; or the mobile terminal 100 can regularly update the room and device information, and then prompt the user that the room and device have been updated and ask the user to confirm whether there is an error. If there is an error, the user can be supported to manually change it, otherwise, the latest changed division result is automatically adopted, etc. In the embodiment of the present application, the room information that the mobile terminal 100 supports the user to change includes but is not limited to: room name, equipment contained in the room; the device information that the mobile terminal 100 supports the user to change includes but is not limited to: device name, room to which the device belongs.

It is worth noting that the sweeping robot is a public device, but the position of the sweeping robot is mobile. Therefore, in the process of the sweeping robot obtaining the location of the home device, it also locates itself, and its location may be any room in the home. Therefore, when the sweeping robot divides the results according to the room layout information and the device location information, the sweeping robot may be divided into any room in the home. When the room where the sweeping robot is divided is the private room of a family member, other family members cannot use the sweeping robot. In order to avoid this problem, after the mobile terminal 100 automatically divides the room and the device type, the mobile terminal 100 also supports the user to manually divide the device type. In other words, based on the automatic division results, the user can also move the device in any private room to the public area for use by all family members; or move the device in any public area to a private room for use only by the owner of the private room, and so on.

In the embodiment itself, the devices that the user can re-divide the room with include, in addition to the sweeping robot mentioned above, vacuum cleaners, Bluetooth speakers, massagers, microphones, weighing scales, air purifiers, electric fans, desk lamps, and other household devices that are convenient for users to move around.

It can be seen that in the embodiment of the present application, the mobile terminal 100 can combine the two methods described above, namely, manual division by the user and automatic division by the mobile terminal 100, to divide the room and the devices in the room. In this way, for some home devices whose locations in the home may change but are shared by family members, the following situations can be avoided:

When only the method of automatically dividing rooms and devices is used, when the sweeping robot collects family room information and device information, some public home devices may appear in a family member's private space, such as a bedroom. This will cause the final division result to be incorrect, making other family members unable to use the public home devices.

When only the manual division of rooms and devices is used, since there are dozens of devices in the home, this will bring a heavy operational and memory burden to the user.

In the embodiment of the present application, the mobile terminal 100 can be referred to as the first device, and the sweeping robot can be referred to as the third device. The information sent by the sweeping robot to the mobile terminal 100 to indicate the room to which the household appliances contained in Xiaozhong's home belong can also be referred to as the first information, and the household appliances can also be referred to as the second device. In addition, Xiaozhong's home can also be referred to as the first space, and the rooms contained in the first space can be referred to as the first area. Next, Figures 7A-7H are introduced to exemplify the relevant operation interfaces for users to add family members.

Adding family members means establishing a family group with the mobile terminals of the family members and the mobile terminal 100-1 of the small and medium-sized enterprise. Family members can log in to the "smart home" in their own mobile terminals to manage and control the family.

As shown in FIG. 7A , mobile terminal 100 - 1 detects an operation on home name option 611 in user interface 61 , and in response to the operation, mobile terminal 100 - 1 displays user interface 71 shown in FIG. 7B .

FIG. 7B exemplarily shows the details page of Xiaozhong's home.

As shown in FIG. 7B , the user interface 71 includes: a family name option 711, a family member option 712, a device management option 713, a classification management option 714, and a wallpaper thumbnail display bar 715. Among them, the family name option 711 also corresponds to the display of the name of the family, such as "Xiao Zhong's Home", and the user can modify the family name by clicking the family name option 711. Among them, the family member option 712 also corresponds to the display of the number of family members, for example, "1 member" is displayed at this time, and the user can add family members, set the roles of family members, rooms, etc. by clicking the family member option 712. Among them, the device management option 713 and the classification management option 714 also correspond to the display of the number of family devices, for example, "16 devices" and "7 rooms" are displayed at this time, and the user can add or delete family devices, rooms, etc. by clicking the device management option 713 and the classification management option 714 respectively. Among them, the wallpaper thumbnail display bar 715 displays thumbnails of multiple wallpapers, and the user can preview the wallpaper by clicking any of the thumbnails, and apply the wallpaper in the "Smart Home" APP.

The mobile terminal 100 - 1 may detect an operation of the user clicking on the family member option 712 shown in FIG. 7B . In response to this operation, the mobile terminal 100 - 1 may display the user interface 72 of the family member detail page shown in FIG. 7C .

FIG. 7C exemplarily shows the detail page of Xiaozhong’s family members.

As shown in FIG. 7C , the user interface 72 includes: a member sub-option 721 and an add member control 722. The sub-option 721 also corresponds to the role of the member in the family, such as "creator". The definition of the creator can be specifically referred to the description of the related concepts involved in the family in the previous text, which will not be repeated here. The user can transfer the identity of the family "creator" to other members of the family by clicking the sub-option 721.

The mobile terminal 100 - 1 may detect that the user clicks the add member control 722 shown in FIG. 7C . In response to the operation, the mobile terminal 100 - 1 displays the user interface 73 of the add member details page shown in FIG. 7D .

FIG. 7D exemplarily shows a detail page for adding a member.

As shown in FIG7D , the user interface 73 includes: a close icon 731A, a completion icon 731B, a scan code to add members option 732, an add member option through a Huawei account option 733, and an add member option through a social application 734, etc. The user can add members to the family of Xiaozhong by clicking any of the above three options.

Taking the first method of adding members as an example, the mobile terminal 100-1 can detect the user's operation of clicking the scan code to add members option 732 shown in Figure 7D. In response to this operation, the mobile terminal 100-1 will display the scan user interface 74 shown in Figure 7E.

FIG. 7E exemplarily shows a scan interface.

As shown in FIG7E , the user interface 74 includes: a code scanning area 741. The code scanning area 741 is used to display the image captured by the camera, such as the image of Xiaohua's mobile terminal, and at this time Xiaohua's mobile terminal 100-2 displays the QR code of Xiaohua's smart home APP. When the mobile terminal 100-1 recognizes the QR code, it will jump to display Xiaohua's information as shown in FIG7F.

As shown in FIG. 7F , in addition to the information shown in FIG. 7D , the user interface 73 also displays the most recently scanned user, for example, option 735 of member Xiaohua.

Mobile terminal 100-1 can detect the operation of the user clicking option 735 shown in Figure 7F. In response to this operation, mobile terminal 100-1 selects member Xiaohua. Then mobile terminal 100-1 can detect the operation acting on icon 731B. In response to this operation, mobile terminal 100-1 will jump to user interface 72 that displays the family member details page as shown in Figure 7G.

As shown in FIG. 7G , in addition to the information shown in FIG. 7C , the user interface 72 also displays options for the added users, for example, option 723 for member Xiaohua.

Afterwards, the user can also add members through the method described in Figures 7A-7G above, and eventually add all the members of Xiaozhong's family, such as Xiaoming, Xiaohong, Xiaoai, Xiaoyu, Xiaoxue, etc. mentioned above, to Xiaozhong's home. See the example in Figure 7H for details, and I will not repeat it here.

As shown in Figure 7H, Figure 7H exemplarily shows a user interface diagram after Xiaozhong adds all family members to the family. It can be seen that at this time, the family member details page displays: option 721 of member Xiaozhong, option 723 of member Xiaohua, option 724 of member Xiaoming, option 725 of member Xiaohong, option 726 of member Xiaoai, option 727 of member Xiaoyu, option 728 of member Xiaoxue, etc. It is worth noting that except for the role displayed as "creator" corresponding to option 721 of member Xiaozhong, the roles of the remaining newly added members are all defaulted to "ordinary users". Regarding the definition of "ordinary users", please refer to the description of the relevant concepts involved in the family in the previous text, which will not be repeated here. Users can set the roles and rooms of family members by clicking on the options of the above-mentioned family members.

7I-7T exemplarily illustrate the operation interface for setting roles and rooms of family members.

Mobile terminal 100-1 may detect the user's operation of clicking option 723 of member Xiaohua shown in FIG7H. In response to this operation, mobile terminal 100-1 displays user interface 75 of the member information details page corresponding to Xiaohua as shown in FIG7I.

FIG. 7I exemplarily shows the member information details page corresponding to Xiaohua.

As shown in FIG7I , the user interface 75 includes: Huawei account 751, member name 752, member role 753, member room 754, remove member control 755, etc. It is worth noting that the member role 753 defaults to ordinary user, and the member room 754 defaults to all rooms. The user can set Xiaohua's role and room by clicking the member role 753 and member room 754 options.

When the mobile terminal 100 - 1 detects that the user clicks the option 753 of the member role as shown in FIG. 7I , in response to the operation, the mobile terminal 100 - 1 will display the user interface 76 of the member role detail page as shown in FIG. 7J .

FIG. 7J exemplarily shows the member information details page corresponding to Xiaohua.

As shown in FIG. 7J , the user interface 76 includes: a creator option 761, an administrator option 762, an ordinary user option 763, a child; an elderly option 764, a close icon 765A, and a completion icon 765B, etc. In addition, the options of the above-mentioned roles also correspond to the permissions corresponding to the different roles. For example, the creator has the highest permissions, and can manage all family members, including administrators, ordinary users, children, the elderly, etc., and can also manage and use all home devices. The administrator has permissions second only to the creator, and can manage all family members, including administrators, ordinary users, children, the elderly, etc., and can also manage and use all home devices.

Mobile terminal 100-1 can detect the operation of a user clicking on option 762 shown in Figure 7J. In response to this operation, mobile terminal 100-1 selects the member role as "administrator". Then mobile terminal 100-1 can detect the operation acting on icon 765B. In response to this operation, mobile terminal 100-1 will jump to user interface 75 displaying the member information details page as shown in Figure 7K.

It is understandable that the classification of member roles shown in FIG. 7J is only an example and should not constitute a limitation on the embodiment itself. In other embodiments of the present application, the mobile terminal may further divide member roles into more or fewer types, for example, it may further include family visitors, or it may further subdivide children and the elderly into two roles, which is not limited in the embodiments of the present application.

As shown in FIG. 7K , the member role displayed on the user interface 75 is changed from “ordinary user” as shown in FIG. 7I to “administrator”.

Mobile terminal 100-1 may detect an operation of a user clicking option 754 shown in FIG. 7K. In response to this operation, mobile terminal 100-1 may display user interface 77 of a member room details page as shown in FIG. 7L.

FIG. 7L exemplarily shows the member room details page corresponding to Xiaohua.

As shown in FIG7L , the user interface 77 includes: a close icon 771A, a completion icon 771B, a list item 653A, a list item 772, a list item 773, a list item 774, a list item 775, a list item 776, a list item 777, a list item 778, and a list item 779, etc. The rooms corresponding to the above list items are bedroom A, bedroom B, bedroom C, living room, kitchen, study, gym, entertainment room, etc. When the number of list items exceeds a preset number, such as 10, the user can browse the list items not shown by sliding the user interface 77 up and down.

Mobile terminal 100-1 can detect the user's click operation on list item 772, list item 775, list item 776, list item 777, list item 778 and list item 779 shown in Figure 7L. In response to this operation, the member rooms selected by mobile terminal 100-1 are bedroom B, living room, kitchen, study, gym, entertainment room, etc., and then mobile terminal 100-1 can detect the operation acting on icon 771B. In response to this operation, mobile terminal 100-1 will jump to user interface 75 displaying the member information details page as shown in Figure 7M.

As shown in FIG. 7M , the member rooms displayed on the user interface 75 are changed from the original default “all” in FIG. 7K to “bedroom B, living room, kitchen…”.

Figures 7I to 7M above introduce exemplary operations for setting Xiaohua's family role. Since Xiaohua is the mother of the family, she is set as the administrator so that Xiaohua can manage family devices and family members. Next, take grandfather Xiaoming as an example. Since grandfather is old and has difficulty in moving, Xiaozhong wants to set grandfather Xiaoming's member role to "child; elderly". The specific examples are as follows.

Mobile terminal 100-1 may detect an operation of the user clicking on the return control shown in FIG. 7M. In response to this operation, mobile terminal 100-1 jumps to user interface 72 displaying the family member details page shown in FIG. 7N.

For the description of the content shown in Figure 7N, please refer to the introduction to Figure 7H above. Specifically, the family member details page at this time displays: option 721 for member Xiaozhong, option 723 for member Xiaohua, option 724 for member Xiaoming, option 725 for member Xiaohong, option 726 for member Xiaoai, option 727 for member Xiaoyu, option 728 for member Xiaoxue, etc. It is worth noting that except for the role displayed as "creator" corresponding to option 721 of member Xiaozhong, the roles of the remaining newly added members are all defaulted to "ordinary users". For the definition of "ordinary users", please refer to the previous description of the relevant concepts related to the family, which will not be repeated here. Users can set the roles and rooms of family members by clicking on the options of the above family members.

Mobile terminal 100-1 may detect the user's operation of clicking option 724 of member Xiao Ming shown in FIG. 7N . In response to this operation, mobile terminal 100-1 displays user interface 75 of the member information details page corresponding to Xiao Ming as shown in FIG. 7O .

FIG. 7O exemplarily shows the member information details page corresponding to Xiao Ming.

As shown in FIG7O , the user interface 75 includes: Huawei account 751, member name 752, member role 753, member room 754, remove member control 755, etc. It is worth noting that the member role 753 defaults to ordinary user, and the member room 754 defaults to all rooms. The user can set Xiaohua's role and room by clicking the member role 753 and member room 754 options.

When the mobile terminal 100 - 1 detects that the user clicks the option 753 of the member role as shown in FIG. 7O , in response to the operation, the mobile terminal 100 - 1 will display the user interface 76 of the member role detail page as shown in FIG. 7P .

FIG. 7P exemplarily shows the member information details page corresponding to Xiao Ming.

As shown in FIG. 7P , the user interface 76 includes: creator option 761, administrator option 762, ordinary user option 763, children; elderly option 764, close icon 765A and completion icon 765B, etc. In addition, the options of the above-mentioned roles also correspond to the permissions corresponding to different roles. For example, the creator has the highest authority, and can manage all family members, including administrators, ordinary users, children, the elderly, etc., and can also manage and use all home devices. The administrator has the authority second only to the creator, and can manage all family members, including administrators, ordinary users, children, the elderly, etc., and can also manage and use all home devices.

Mobile terminal 100-1 can detect the operation of the user clicking on option 764 shown in Figure 7P. In response to this operation, mobile terminal 100-1 selects the member role as "Child; Elderly". Then mobile terminal 100-1 can detect the operation acting on icon 765B. In response to this operation, mobile terminal 100-1 will jump to the user interface 75 that displays the member information details page as shown in Figure 7Q.

As shown in FIG. 7Q , the member roles displayed on the user interface 75 are changed from “ordinary user” as shown in FIG. 7O to “child; elderly”.

Mobile terminal 100-1 may detect an operation of a user clicking option 754 shown in FIG. 7Q. In response to this operation, mobile terminal 100-1 may display user interface 77 of a member room details page as shown in FIG. 7R.

FIG. 7R exemplarily shows the member room details page corresponding to Xiao Ming.

As shown in FIG7R , the user interface 77 includes: a close icon 771A, a completion icon 771B, a list item 653A, a list item 772, a list item 773, a list item 774, a list item 775, a list item 776, a list item 777, a list item 778, and a list item 779, etc. The rooms corresponding to the above list items are bedroom A, bedroom B, bedroom C, living room, study, gym, entertainment room, etc. When the number of list items exceeds a preset number, such as 10, the user can browse the list items not shown by sliding the user interface 65 up and down.

It is worth noting that, as shown in FIG7P above, Xiao Ming's member role is set to "child; elderly", and children and elderly people are more inconvenient to act than teenagers and middle-aged people, so some dangerous behaviors such as using kitchen appliances need to be prohibited. Therefore, option 776 shown in FIG7R, i.e., the kitchen option, cannot be selected by default.

Mobile terminal 100-1 can detect the user's click operation on list item 773, list item 775, list item 777, list item 778 and list item 779 shown in Figure 7R. In response to this operation, the member room selected by mobile terminal 100-1 is bedroom B, living room, kitchen, study, gym, entertainment room, etc., and then mobile terminal 100-1 can detect the operation acting on icon 771B. In response to this operation, mobile terminal 100-1 will jump to the user interface 75 displaying the member information details page as shown in Figure 7S.

As shown in FIG. 7S , the member rooms displayed on the user interface 75 are changed from the original default “all” in FIG. 7Q to “bedroom B, living room, study…”.

After that, the user can also set the roles and rooms of the members in the manner described in Figures 7I-7S above, and finally set the role of all members of Xiaozhong's family, such as the grandmother Xiaohong mentioned above (Xiaohong is younger than Xiaoming and is more mobile), to "Administrator", and the rooms are bedroom B, living room, kitchen, study, gym, and entertainment room; Xiaoai (Xiaoai is still a child) has the role of "Child; Elder", and the rooms are bedroom C, living room, study, gym, and entertainment room; Xiaoyu, Xiaoxue, etc. are temporary family members, so their roles are set to "Ordinary User", and the rooms are only the public areas of the home, such as the living room, kitchen, gym, and entertainment room. Since the study usually holds the valuable documents of the owner, it is not open to temporary users, housekeepers, or tutors.

As shown in Figure 7T, Figure 7T exemplarily shows a user interface diagram of Xiaozhong setting roles for all family members to divide a family into rooms. It can be seen that at this time, the family member details page displays: option 721 of member Xiaozhong, option 723 of member Xiaohua, option 724 of member Xiaoming, option 725 of member Xiaohong, option 726 of member Xiaoai, option 727 of member Xiaoyu, option 728 of member Xiaoxue, etc. It is worth noting that option 721 of member Xiaozhong corresponds to the displayed role of "creator", Xiaohua's role is "administrator", member Xiaoming's role is "children; elderly", member Xiaohong's role is "administrator", Xiaoai's role is "children; elderly", and Xiaoyu and Xiaoxue's roles are "ordinary users". Users can reset or change the role of any family member, as well as the room, etc. by clicking on the above family member options.

It is understandable that, in addition to the above-mentioned Figures 7I-7T, which illustrate how to divide rooms so that members of different roles can only use the devices in one or more rooms belonging to the role, the permission for family members to use any one or more devices in one or more rooms can also be set. In other words, the permission for home devices is not only divided based on all devices in a room, but can also be divided specifically based on a certain device or a certain type of device in the room. The embodiment of the present application does not limit the division of device usage permissions.

After the user enters the registration information in the smart home APP as described in Figures 6A to 6V above, and after the user adds the relevant information of the family members in the smart home APP as described in Figures 7A to 7T above, the electronic device will collect the user's registration information, family member information, etc. That is, the collection of user data mentioned in the data bank system above. The method of collecting user data will not be described here in detail. For details, please refer to the introduction of the data bank system above.

Based on the operation interfaces of adding home devices to Xiaozhong's home and setting the rooms to which the home devices belong as exemplarily shown in FIGS. 6A to 6V above, and the operation interfaces of adding family members to Xiaozhong's home and setting the roles played by the family members and the rooms of the family members as exemplarily shown in FIGS. 7A to 7T above, the following describes a UI embodiment for a user to use home devices.

In the embodiment of the present application, in order to ensure that family members can only use the home devices within the scope of their authority, when the user uses the home device, the user needs to be authenticated to determine whether the user has the authority to use the home device. In this way, it can be prevented that people who do not have the authority to use the home device perform some dangerous operations or malicious operations. For example, in order to prevent the elderly or children from using dangerous electrical appliances, or family visitors from using privacy devices in the home, such as expensive insurance, etc., which may cause losses to the family.

Next, we will take the mobile terminals 100 - 1 of Xiaozhong and Xiaozhong as examples and introduce the initialization operation of the user setting the identity authentication method in conjunction with FIG. 8A to FIG. 8E .

8A-8E are schematic diagrams of an initialization operation interface for a user to set an identity authentication method.

FIG. 8A exemplarily shows the home page of the smart home APP.

As shown in FIG. 8A , the user interface shown in FIG. 8A is similar to the user interface shown in FIG. 6K in the previous text. The previous introduction to FIG. 6K is applicable to the introduction to FIG. 8A and will not be repeated here.

The mobile terminal 100 - 1 may detect that the user clicks the My option 615D in the page navigation bar 615 in FIG. 8A . In response to this operation, the mobile terminal 100 - 1 displays the user interface 81 as shown in FIG. 8B .

FIG8B exemplarily shows the “My” page provided by the Smart Home APP.

As shown in FIG8B , the user interface 81 displays: option 811, option 812, option 813, option 814, option 815, option 816, option 817, and option 818. Option 811 is used to open the "Me" details page, which specifically provides users with services such as account name, password, bound mobile phone number, and password-free authentication; option 812 is used to open the My Family page, which specifically provides users with services such as adding and removing family members; option 813 is used to open the user interface of My Main Device, which specifically provides users with operations to change the main device. Generally, the main device defaults to the device currently logged into the Smart Home APP, that is, the local device; option 814 is used to open the My Other Devices page, which specifically provides users with functions to manage other devices of the user, such as tablets, notebooks, bracelets, headphones, etc.; option 81 Option 5 is used to open the My QR Code page, which provides the user with a QR code that identifies the current Smart Home APP account. Other terminals can add the user identified by the QR code as a family member or friend by scanning the QR code; Option 816 is used to open the user interface for connecting to a third-party platform, which specifically provides the user with platforms provided by other home device manufacturers, operators, etc. that cooperate with the Smart Home APP; Option 817 is used to open the Settings page, which provides the user with services for setting common functions of the Smart Home APP; Option 818 is used to open the Help page, which specifically includes prompts to the user on how to use the Smart Home APP and precautions, etc.

Mobile terminal 100 - 1 may detect an operation of a user clicking on option 811 in FIG. 8B , and in response to the operation, mobile terminal 100 - 1 displays user interface 82 as shown in FIG. 8C .

As shown in FIG8C , the user interface 82 displays: option 821, option 822, option 823, and option 824, etc. The above options correspond to the user's account name, bound mobile phone number, login password corresponding to the account, password-free authentication, etc.

Mobile terminal 100 - 1 may detect an operation of the user clicking option 824 in FIG. 8C , and in response to this operation, mobile terminal 100 - 1 displays user interface 83 as shown in FIG. 8D .

As shown in FIG8D , the user interface 83 displays: option 831, option 832, and option 833, etc., and the above options correspond to fingerprint authentication, face authentication, and voiceprint authentication, etc. The user can open the corresponding authentication method by clicking any one or more of the above options. It is worth noting that FIG8D only exemplifies three commonly used authentication methods. In the embodiment of the present application, the mobile terminal 100-1 can provide more or fewer authentication methods, and the present application does not limit this.

When mobile terminal 100-1 detects that the user has turned on the above-mentioned identity authentication, it can display the user interface shown in Figure 8E. Specifically, options 831, option 832 and option 833 displayed in the user interface 83 correspond to prompt information that has been turned on, respectively, to prompt the user that the identity authentication method has been turned on.

In the embodiment of the present application, the mobile terminal 100-1 can support multiple identity authentication methods, such as: password authentication based on biometrics, behavioral characteristics, and identity authentication. Among them, biometric authentication methods may include: face authentication, fingerprint authentication, palm print authentication, iris authentication, retina authentication, and identity authentication based on photoelectric plethysmography (PPG) signals, etc. Behavioral authentication methods may include: authentication based on gait, handwriting, lip movement, sound, and key strength, etc. Password authentication may also include: character password authentication, pattern password authentication, etc. The embodiment of the present application does not limit the identity authentication methods supported by the mobile terminal 100-1.

After detecting an operation for triggering the mobile terminal 100-1 to perform identity authentication, the mobile terminal 100-1 may, according to the surrounding environment and/or the user status, give priority to enabling the biometric authentication and behavioral authentication methods with an authentication success rate higher than the threshold. Further, when there are multiple biometric authentication and behavioral authentication methods with an authentication success rate higher than the threshold, the mobile terminal 100-1 may first enable the high-priority biometric authentication or behavioral authentication according to the preset priority from high to low. When the high-priority biometric authentication or behavioral authentication fails, the mobile terminal 100-1 then enables the low-priority biometric authentication or behavioral authentication. If any of the biometric authentication or behavioral authentication methods succeeds, the mobile terminal 100-1 determines that the identity authentication is successful. If all of the biometric authentication methods and behavioral authentication methods fail, the mobile terminal 100-1 enables password authentication until the password entered by the user matches the preset password, then the identity authentication is determined to be successful. If they do not match, the identity authentication is determined to have failed.

The above-mentioned biometric authentication and behavior authentication priorities may be set by default in the mobile terminal 100 - 1 or may be customized by the user, and this embodiment of the present application does not impose any limitation on this.

In an embodiment of the present application, when the mobile terminal 100-1 turns off the biometric authentication and behavioral authentication methods with authentication success rates lower than a threshold value, and turns on the biometric authentication and behavioral authentication methods with authentication success rates higher than the threshold value according to the surrounding environment and/or user status, the mobile terminal 100-1 can identify the user identity based on the collected user data corresponding to the enabled authentication method, or transmit the collected user data corresponding to the authentication method to the server 300, which identifies the user identity and sends the identification result to the mobile terminal 100. The embodiment of the present application does not impose any restrictions on this.

In addition, the mobile terminal 100-1 can disable the biometric authentication and behavior authentication methods with authentication success rates lower than the threshold value and enable the biometric authentication and behavior authentication methods with authentication success rates higher than the threshold value according to the surrounding environment and/or user status in the following two ways:

In one implementation, the mobile terminal 100-1 pre-associates and stores identity authentication methods corresponding to different surrounding environments and/or user statuses. When the mobile terminal 100-1 collects data related to the surrounding environment and/or user status, the mobile terminal 100-1 can match the authentication method corresponding to the data.

In another implementation, after the mobile terminal 100-1 collects data related to the surrounding environment and/or user status, it sends the data to the server to request the authentication method to be adopted. The server calculates the authentication success rate corresponding to each authentication method based on the data, and sends the authentication method with a success rate higher than the threshold to the mobile terminal 100-1. The mobile terminal 100-1 turns on the authentication method and turns off other authentication methods.

Next, taking the mobile terminals 100-1 of members Xiaozhong and Xiaozhong, and the mobile terminals 100-3 of members Xiaoming and Xiaoming as examples, combined with Figures 8F-8N, the user interfaces of family members controlling home devices through the smart home APP are exemplified.

Family members can open the smart home APP installed in the mobile terminal 100, click the icon of the home device displayed on the home page, open the corresponding home device details page, view the current status of the home device, and input operations for controlling the home device. Only when the mobile terminal recognizes the identity of the current user and determines that the user has the authority to control the home device, will it control the home device to execute the corresponding function.

It is worth noting that when family members control and use home devices through the smart home APP installed on the mobile terminal 100, the communication distance between the location of the mobile terminal 100 and the home device may be less than the threshold, for example, the mobile terminal 100 is within the home range, or the communication distance between the mobile terminal 100 and the home device may be greater than the threshold, for example, the user is away from home, at the company, or on the way home from the company, etc. When family members control and use home devices through the smart home APP installed on the mobile terminal 100, when the communication distance between the location of the mobile terminal 100 and the home device 200 is less than the threshold, the mobile terminal 100 can directly control the home device 200, or it can also control the home device 200 through the server 300. When family members control and use home devices through the smart home APP installed on the mobile terminal 100, when the communication distance between the location of the mobile terminal 100 and the home device 200 is greater than the threshold, the mobile terminal 100 must control the home device 200 through the server 300. In other words, the smart home APP supports remote control of home devices 200.

First, FIG. 8F to FIG. 8I exemplarily illustrate schematic diagrams of operating interfaces for family members to use home devices.

FIG8F exemplarily shows the home page of the smart home APP installed on the mobile terminal 100 - 1 of Xiaozhong.

As shown in FIG. 8F , the user interface 61 shown in FIG. 8F is similar to the user interface 61 shown in FIG. 6K above. The above description of FIG. 6K is applicable to the description of FIG. 8F and will not be repeated here.

The mobile terminal 100 - 1 may detect that the user clicks on the device icon corresponding to the rice cooker in the device display area 614 in FIG. 8F . In response to the operation, the mobile terminal 100 - 1 displays the user interface 84 as shown in FIG. 8G .

FIG8G exemplarily shows a device details page corresponding to an electric rice cooker.

As shown in FIG8G , the user interface 84 shown in FIG8G displays a series of options for controlling the electric rice cooker, such as a switch option 841, a mode option 842, a reservation time option 843, etc. Among them, the switch option 841 corresponds to a control for turning on the electric rice cooker, the mode option 842 corresponds to a control for switching the working mode of the electric rice cooker, and the reservation time option 843 corresponds to a control for setting the working time of the electric rice cooker.

Since the user Xiaozhong of the mobile terminal 100-1 is the creator of the family, Xiaozhong has the highest authority of the family, that is, Xiaozhong can manage and use all home devices including the rice cooker described above. However, in order to prevent other users other than Xiaozhong from using Xiaozhong's mobile terminal 100-1 to control the electronic devices at home, the mobile terminal 100-1 will turn on the identity authentication function to authenticate the current user when it detects the user input triggering the operation of the mobile terminal 100-1 to control the home device. For example, the mobile terminal 100-1 can detect the operation of the user clicking the control corresponding to option 841 shown in Figure 8G to turn on the rice cooker. In response to the operation, the mobile terminal turns on the identity authentication function. And determine whether the identity of the current user has the authority to use the electronic device, that is, the desk lamp. If it has the authority, it will respond normally and execute the operation input by the user; when the mobile terminal 100-1 detects that the identity of the current user does not have the authority to use the rice cooker, it prompts the current user that the control and use of the rice cooker cannot be executed, and the operation input by the user will no longer be executed.

In other embodiments of the present application, the operation that triggers the mobile terminal 100-1 to start identity authentication is not limited to the above-mentioned operation for controlling home devices, but may also be the mobile terminal 100-1 detecting that the user opens the operation for displaying the home device details page, or the user periodically starts identity authentication when running the smart home APP, continuously authenticates the user, etc. The present application does not limit this.

Assuming that the mobile terminal 100 - 1 recognizes that the current user is not Xiaozhong, a prompt message as shown in FIG. 8H will pop up.

As shown in FIG8H , the user interface 84 shown in FIG8H pops up a prompt message 845 to prompt the user that “the owner is not recognized, and the operation fails.” In the embodiment of the present application, the prompt message 845 popped up in FIG8H may also be referred to as the first prompt message.

Assuming that the mobile terminal 100-1 recognizes that the current user is Xiaozhong himself, it will determine whether Xiaozhong has the authority to use the rice cooker. Since Xiaozhong is the home creator and has the highest authority, that is, he can use all home devices, the mobile terminal 100-1 will pop up a prompt message as shown in Figure 8I.

As shown in FIG8I , the user interface 84 shown in FIG8I pops up a prompt message 846 to prompt the user that “the operation is successful, and the rice cooker has been turned on for you”.

In the embodiment of the present application, the mobile terminal 100-1 can be referred to as the first device; the rice cooker belongs to the second electronic device, and the second electronic device also includes the various household appliances provided above, such as televisions, cameras, electric curtains, table lamps, electric kettles, refrigerators, air conditioners, air purifiers, smoke stoves, etc.

In the embodiment of the present application, the room where the rice cooker is located is a kitchen, which can also be referred to as the first area.

In the embodiment of the present application, the first operation is the operation for starting the rice cooking function of the rice cooker as exemplified in FIG. 8G above.

In the embodiment of the present application, the cooking function for starting the rice cooker can also be referred to as the first function for starting the second electronic device. In the embodiment of the present application, Xiaozhong can also be referred to as the first user, and Xiaoai can also be referred to as the second user.

8J to 8N exemplarily show schematic diagrams of the operating interface for family member Xiao Ming to use home devices.

FIG8J exemplarily shows the home page of the smart home APP installed on Xiao Ming's mobile terminal 100 - 3.

As shown in FIG8J , the user interface displayed by mobile terminal 100 - 3 is similar to user interface 61 displayed by mobile terminal 100 - 1 shown in FIG6K and FIG8F above. The above description of FIG6K is applicable to the description of FIG8J and will not be repeated here.

It is worth noting that the icons of some household devices displayed in the device display area shown in FIG8J are not the same as the icons of some household devices displayed in the device display area 614 shown in FIG8F. Specifically, the display status of the icons corresponding to the electric kettle, rice cooker, TV A, air purifier A, etc. shown in FIG8J is in grayscale, which is used to prompt the user that the household devices corresponding to these icons are not under the control of the user. This is because Xiao Ming's role in Xiao Zhong's home is "child; elderly". Therefore, Xiao Ming has no right to use the dangerous household equipment in the kitchen; in addition, because the available rooms in Xiao Ming's room division do not include Xiao Zhong and Xiao Hua's bedroom A, and Xiao Ai's bedroom C, Xiao Ming has no right to use the equipment in bedroom A and bedroom C, such as TV A, air purifier A, etc. However, Xiao Ming can use these household devices after authorization by the family creator and/or administrator, as follows:

Mobile terminal 100-3 can detect the operation of a user clicking on an icon corresponding to any household device in the device display area in FIG8J, such as clicking on the device icon corresponding to a rice cooker. In response to this operation, mobile terminal 100-3 displays a user interface as shown in FIG8K.

As shown in FIG. 8K , the user interface displayed by the mobile terminal 100 - 3 is similar to the user interface 84 shown in FIG. 8G above. The above description of FIG. 8G is applicable to the description of FIG. 8K and will not be repeated here.

Mobile terminal 100-3 detects user input to trigger the operation of mobile terminal 100-3 controlling home appliances, for example, it detects the operation of the user clicking the control corresponding to the switch option in Figure 8K to turn on the rice cooker. In response to this operation, mobile terminal 100-3 displays the user interface shown in Figure 8L.

As shown in FIG8L , the user interface shown in FIG8L displays a prompt window, and the prompt window displays information for prompting the user that the user does not have the authority for the above operation and asking the user whether to request the authority from others. The prompt information displayed in the prompt window can also be called the first prompt information. The "OK" control shown in FIG8L can also be called the first control.

The mobile terminal 100-3 detects the operation acting on the determined control as shown in FIG8L. In response to the operation, the mobile terminal 100-3 may send a permission request, i.e., a first request, to one or more mobile terminals respectively equipped with the family creator and/or the family manager. The mobile terminals here include any one or more of the following: mobile phones, bracelets, headphones, notebooks, tablet computers, etc. For example, the mobile terminal 100-3 sends a permission request to the mobile terminal 100-1 equipped with the small middle, and the mobile terminal 100-1 equipped with the small middle will display a prompt message as shown in FIG8M.

As shown in FIG8M , the user interface displayed on the mobile terminal 100 - 1 has a prompt window, in which information is displayed for prompting Xiao Zhong and Xiao Ming to request permission to operate the rice cooker.

The mobile terminal 100-1 detects an operation input for authorizing Xiao Ming, such as clicking the operation of the consent control in the prompt window in FIG8M. In response to the operation, the mobile terminal 100-1 sends an authorization message to the mobile terminal 100-3. After receiving the authorization information, the mobile terminal 100-3 displays the user interface shown in FIG8N. In other embodiments of the present application, after the mobile terminal 100-1 detects the operation of clicking the consent control in the prompt window in FIG8M, it is also necessary to authenticate the user. For example, after detecting that Xiao Zhong's face, fingerprint, or voiceprint authentication has passed, the user interface shown in FIG8N will be displayed. The embodiment of the present application does not limit the method of Xiao Zhong's authorization.

As shown in Fig. 8N, in the user interface displayed by mobile terminal 100-3, the control for turning on the rice cooker corresponding to the switch option is switched from the closed state in Fig. 8K to the open state. In addition, a prompt window is also displayed, which is used to prompt Xiao Ming that Xiao Zhong has agreed to the permission request and has turned on the rice cooker for you.

Figures 8J-8N above are only schematic diagrams of the operation interface of the home devices outside the scope of the family member Xiao Ming's use authority. Next, Figures 8O-8Q are combined to exemplarily illustrate the operation interface of the home devices within the scope of the family member Xiao Ming's use authority.

As shown in Figure 8O, the mobile terminal 100-3 detects the user's operation on the icon of the home device within the scope of authority, and in response to the operation, opens the detail page corresponding to the home device. For example, the mobile terminal 100-3 detects the operation on the icon of the desk lamp, and in response to the operation, the mobile terminal 100-3 displays the user interface shown in Figure 8P.

As shown in FIG8P , the details page corresponding to the desk lamp displayed by the mobile terminal 100 - 3 includes: switch options, mode options, brightness options, etc. Among them, the switch option has a control for turning on the desk lamp, the mode option has a control for switching the working mode of the desk lamp, and the brightness option has a slider for adjusting the brightness.

When the mobile terminal 100-3 detects that the user input triggers the operation of the mobile terminal 100-3 to control the home device, the identity authentication function will be turned on to authenticate the current user. For example, the mobile terminal 100-3 can detect that the user clicks the switch option shown in FIG8P to turn on the control of the table lamp. In response to the operation, the mobile terminal 100-3 turns on the identity authentication function and determines whether the identity of the current user has the authority to use the electronic device, i.e., the table lamp. If the authority is available, the mobile terminal 100-3 responds normally and executes the operation input by the user; when the mobile terminal 100-3 detects that the identity of the current user does not have the authority to use the rice cooker, it prompts the current user that the rice cooker cannot be used, and no longer executes the operation input by the user.

In other embodiments of the present application, the operation that triggers the mobile terminal 100-3 to start identity authentication is not limited to the above-mentioned operation for controlling home devices, but can also be the mobile terminal 100-3 detecting that the user opens the operation for displaying the home device details page, or the user periodically starts identity authentication when running the smart home APP, continuously authenticates the user, etc. The present application does not limit this.

Assuming that the mobile terminal 100-3 recognizes that the current user is Xiao Ming himself, it will determine whether Xiao Ming has the authority to use the desk lamp. Since Xiao Ming's rooms include bedroom B, living room, study, gym, entertainment room, etc., and the desk lamp is deployed in the study, Xiao Ming has the right to use the desk lamp. The mobile terminal 100-3 will pop up a prompt message as shown in Figure 8Q.

As shown in FIG8Q , the mobile terminal 100 - 3 pops up a prompt message to prompt the user that “the operation is successful and the desk lamp has been turned on for you”.

It is worth noting that when family members control and use home devices through the smart home APP installed on the mobile terminal 100, the communication distance between the location of the mobile terminal 100 and the home device may be less than the threshold, for example, the mobile terminal 100 is within the home range, or the communication distance between the mobile terminal 100 and the home device may be greater than the threshold, for example, the user is away from home, at the company, or on the way home from the company, etc. When family members control and use home devices through the smart home APP installed on the mobile terminal 100, when the communication distance between the location of the mobile terminal 100 and the home device 200 is less than the threshold, the mobile terminal 100 can directly control the home device 200, or it can also control the home device 200 through the server 300. When family members control and use home devices through the smart home APP installed on the mobile terminal 100, when the communication distance between the location of the mobile terminal 100 and the home device 200 is greater than the threshold, the mobile terminal 100 must control the home device 200 through the server 300. In other words, the smart home APP supports remote control of home devices 200.

The above-mentioned determination of the user's control authority over home devices based on the user identity information in FIG8A-FIG8Q reflects the processing of user data mentioned in the above-mentioned data bank system, that is, establishing an association between the user identity and the room information of the user in the home space and the device information in the room, thereby determining the control authority over the home devices based on the user's identity. The method of user data processing will not be described in detail here, and the details can be referred to the above-mentioned introduction to the data bank system.

Next, in conjunction with Figures 8R to 8V, a schematic diagram of the operating interface for controlling home devices according to intelligent control rules is introduced.

In some embodiments of the present application, the mobile terminal 100 will also learn a set of intelligent control rules based on the user's living habits. When the mobile terminal 100 detects that the user inputs a control instruction to operate a certain home device 200, the mobile terminal 100 will determine whether to continue to control the corresponding home device 200 to perform the corresponding operation according to the intelligent rules, and display information to prompt the user of the execution result.

In other embodiments of the present application, the mobile terminal 100 will learn a set of intelligent control rules based on the user's living habits and upload them to the server 300. When the server 300 receives the control instruction sent by the mobile terminal 100, it will determine whether to continue controlling the corresponding home device 200 to perform the corresponding operation based on the above-mentioned intelligent control rules, and feed back the execution result to the mobile terminal 100, so that the mobile terminal 100 displays information for prompting the user of the execution result.

In the embodiment of the present application, the above intelligent control rules can be any one or more of the factors such as the distance between the user who initiates the command and the home, the user's current location, the current time, the type of service corresponding to the control command, whether there are other members in the home, or the roles played by other members, etc., to formulate a set of intelligent and safe control rules. The following is a detailed description of the intelligent control rules with several simple examples.

Referring to FIG. 8R , FIG. 8R exemplarily shows a schematic diagram of a daily life route of a small child that is learned by the mobile terminal 100 - 1 .

As shown in Figure 8R, Figure 8R exemplarily shows the location information of Xiaozhong's home, company, and Xiaozhong's daughter Xiaoai's school, where the first route is the route that Xiaozhong must take when going to and from get off work, and the second route is the route that Xiaozhong must take when picking up Xiaoai to and from school. Xiaozhong's life timeline is: from Monday to Friday, he sends his daughter Xiaoai to school around 8:00-8:30 in the morning, and then goes to work at the company. After getting off work around 5:30 in the afternoon, he goes to the school to pick up his daughter after school, and then drives home. On weekends, Xiaozhong's life timeline is not fixed. Sometimes Xiaozhong may rest at home, or he may go on a date with his wife Xiaohua, or he may take Xiaoai's grandfather Xiaoming to the hospital for a follow-up visit, etc.

Based on the above-learned living habits of Xiaozhong, mobile terminal 100-1 can customize a set of intelligent control rules for Xiaozhong. The intelligent control rules are based on Xiaozhong's living habits and any one or more of the factors such as the distance from the user to home, the user's current location, the current time, the type of service corresponding to the control instruction, whether there are other members at home, or the roles played by other members, etc., to formulate a set of intelligent and safe control rules. The following is a detailed description of the intelligent control rules with several simple examples.

Example 1: When there are no other family members, or other family members play the role of "elderly, children", if the user who issues the operation instruction is beyond the preset range, that is, exceeds the first value, or the user's geographical location is not near the first route or the second route learned by the mobile terminal 100-1, for example, the user is at the first location as shown in FIG8R, because the distance from the first location to Xiaozhong's home exceeds the first value, the server 300 refuses to control the home device 200 to perform the corresponding operation. In this way, it can be avoided that in the future for a long time, the user may not rush home, and there is no one at home, which will cause waste of home appliance resources on the one hand, and bring potential safety hazards to the family on the other hand.

Example 2: When there are no other family members, or the roles of other family members are "elderly or children", if the user who issues the operation instruction is within the preset range from home or the user's geographical location is near the first route or the second route learned by the mobile terminal 100-1, for example, the user is at the second location as shown in Figure 8R, because the distance from the second location to the home is less than the first value, the server 300 controls the home device 200 to perform the corresponding operation according to the user's control instruction.

Example 3: When there are other members in the family and the roles played by other members are not "elderly or children", if the user who issues the operation instruction is beyond the preset range of home or the user's geographical location is not near the first route or the second route learned by the mobile terminal 100-1, for example, the user is at the first location as shown in Figure 8R, then the server 300 controls the home device 200 to perform the corresponding operation according to the user's control instruction.

Example 4: When there are no other family members at home during the daytime on any day from Monday to Friday, if the user who issues the operation instruction is at work, and the service type corresponding to the instruction is, for example, turning on the light, air conditioner, TV, etc., which consumes power, the server 300 refuses to control the home device 200 to perform the corresponding operation. In this way, it can avoid the waste of home appliance resources and potential safety hazards caused by no one at home. Implement healthy, safe, low-carbon, and convenient personalized home life.

Example 5: When there are no other members at home during the day on any day from Monday to Friday, if the user who issues the operation instruction is at the company, but the business type corresponding to the instruction is to open/close windows, turn off lights, turn off air conditioners, turn off TVs and other power-consuming home devices 200, then the server 300 controls the home device 200 to perform the corresponding operation.

8S-8V , which exemplarily illustrate schematic diagrams of user interfaces for controlling home devices according to smart control rules.

As shown in Figures 8S to 8T, the mobile terminal 100-1 detects that Xiaozhong inputs an operation to turn on the desk lamp at home. In response to the operation, the mobile terminal 100-1 can obtain Xiaozhong's current location and time information. For example, Xiaozhong is at the first location shown in Figure 8R at this time, and the time is 12:00 noon on Friday. In addition, the mobile terminal 100-1 can also request the server 300 for information about the members of the home. For example, if it is learned from the server 300 that there are no other members in the home, the mobile terminal 100-1 refuses to control the desk lamp to be turned on through the server 300, and displays the prompt message shown in Figure 8U.

As shown in FIG8U, a window 854 is displayed in the user interface 85, and a prompt message is displayed in the window 845, for example, to prompt the user that "no one is detected at home, it is recommended not to turn on the light". If the user still wants to turn on the light, he can click the control 845A in the window. In response to this operation, the mobile terminal 100-1 controls the turning on of the light through the server 300. If the user does not intend to turn on the light temporarily, he can click the control 845B. In response to this operation, the mobile terminal 100-1 closes the window 854. In other embodiments of the present application, the window 854 where the above prompt message is located may not display the control "got it", and the user can act on a place outside the window 854, or when the window display time exceeds a preset time, such as 30 seconds, the mobile terminal 100-1 can close the window 854.

Assuming that the user still wants to turn on the light at this time, the mobile terminal 100-1 can detect the operation acting on the control 845A. In response to the operation, the mobile terminal 100-1 sends an instruction for turning on the desk lamp to the server 300. The server 300 controls the desk lamp to turn on according to the instruction. The server 300 also feeds back the execution result of turning on the desk lamp to the mobile terminal 100. The mobile terminal 100 displays the prompt information shown in Figure 8U according to the feedback message.

The user interface 85 shown in FIG8V displays a window 855, which displays prompt information corresponding to the execution result, such as "The desk lamp has been turned on for you. Since there is no one at home, it is recommended that you withdraw the operation of turning on the light", and control 855A. Control 855A is used to withdraw the previous operation of turning on the light. In addition, the switch control corresponding to option 851 is switched to the off state.

Assuming that the user accidentally touches the control 854A shown in FIG8U, causing the desk lamp to turn on, or the user regrets turning on the desk lamp, the user can also withdraw the above operation. That is, the mobile terminal 100-1 can detect the operation acting on the control 855A shown in FIG8V, and in response to the operation, the mobile terminal 100-1 controls the desk lamp to turn off through the server 300. At this time, the user interface 85 displayed by the mobile terminal is shown in FIG8W, and the switch control corresponding to option 851 is switched to the off state.

Alternatively, the user may also operate at a location other than the window 855 shown in FIG. 8V , or when the display time of the window 855 exceeds a preset time, such as 30 seconds, the mobile terminal 100 - 1 may close the window 855 .

It is understandable that the form in which the prompt information of the execution result is output by the mobile terminal 100 is not limited to the window 854 shown in FIG. 8U , and the prompt information of the execution result can also be output by voice broadcast, etc.

It is worth noting that in addition to displaying the execution result information as shown in FIG8U at the mobile terminal 100-1, if the mobile terminal 100-1 learns that the user is driving, it can also control the onboard computer to output the prompt information of the execution result as shown in FIG8X, so that the user can know the execution result in time. Alternatively, in other scenarios, other mobile terminals such as bracelets, headphones, and tablets can also be controlled to output the prompt information of the execution result at the same time.

The mobile terminal introduced above generates intelligent control rules based on the user's life trajectory, where the user's life trajectory related information involves the opening of user data mentioned in the data bank system mentioned above, which is specifically open to other applications across applications. For example, map navigation apps can open information such as the user's travel trajectory to smart home applications. The method of opening user data will not be described here in detail, and you can refer to the introduction of the data bank system in the previous article for details.

In other embodiments of the present application, the mobile terminal 100 can also combine the method of controlling home appliances according to user identity shown in Figures 8F-8Q above with the method of controlling home appliances according to intelligent control rules shown in Figures 8R-8X above, that is, when the mobile terminal 100 detects or the server 300 receives a control instruction sent by the user, the mobile terminal 100 or the server 300 first performs identity authentication, and when it is recognized that the user has the authority to control the home device 200, the mobile terminal 100 or the server 300 further determines whether to execute the operation corresponding to the control instruction according to the intelligent control rule. The UI interface will not be described in detail here.

In some embodiments of the present application, the mobile terminal can generate an electronic fence. When the mobile terminal detects that the user has left the electronic fence, the mobile terminal can output a second prompt message to prompt the user that he has left the safe area; or, the second message can be sent to the terminal devices of other family members and the furniture devices in the home in the same smart home account as the mobile terminal, which can trigger the terminal devices of other family members and the fixed devices in the home to initiate an alarm, thereby achieving the purpose of protecting the user's personal safety. Next, taking the example of parents setting an electronic fence for their children, the method of setting an electronic fence and the mechanism of triggering an alarm based on the electronic fence are introduced.

In an achievable manner, the mobile terminal can automatically generate an electronic fence according to the user's life trajectory within a second time period, such as within a week. Specifically, Xiao Ai's mobile terminal 100-6 can turn on the electronic fence function by default, or turn on the electronic fence function in response to user operation. Afterwards, the mobile terminal 100-6 can obtain Xiao Ai's activity trajectory and generate an electronic fence based on the activity trajectory. In conjunction with Figure 8R introduced above, Xiao Ai's activity trajectory usually includes Xiao Zhong's home, Xiao Ai's school, and the round-trip route between Xiao Zhong's home and Xiao Ai's school. When the mobile terminal 100-6 detects that Xiao Ai is active at home, the mobile terminal 100-6 can set the range of the community to which Xiao Zhong's home belongs to electronic fence 1 in combination with the map; when the mobile terminal 100-6 detects that Xiao Ai is active anywhere in the school, the mobile terminal 100-6 can set the range of Xiao Ai's school to electronic fence 2 in combination with the map. It can be understood that the ranges of electronic fence 1 and electronic fence 2 shown in Figure 8R are only examples. Each time the mobile terminal 100-6 locates Xiao Ai's position, it is actually a fixed point in the map. The position of Xiao Ai that the mobile terminal 100-6 continuously obtains within a preset time may be the same fixed point, or may be multiple different but adjacent fixed points. The processor of the mobile terminal 100-6 can generate an electronic fence based on the one fixed point, or the area composed of multiple fixed points, combined with its position in the map. The boundary of Xiao Ai's activity area corresponds to the electronic fence, but the area of the electronic fence is usually greater than or equal to the boundary of the actual activity area of Xiao Ai. The embodiment of the present application does not limit the size and shape of the electronic fence. Electronic fence 1 and electronic fence 2 together constitute an electronic fence domain, and the routes between each electronic fence in the electronic fence domain also belong to the scope included in the electronic fence domain. Therefore, Xiao Ai's round-trip route between home and school, that is, the second route, is the necessary route between electronic fence 1 and electronic fence 2, and this route belongs to Xiao Ai's electronic fence domain.

In the embodiment of the present application, the electronic fence 1, the electronic fence 2 and the second route between the electronic fence 1 and the electronic fence 2 may also be referred to as a first electronic fence.

In another achievable manner, the user can actively set the electronic fence domain. Specifically, Xiao Ai's father Xiao Zhong can input the third operation for setting Xiao Ai's electronic fence in Xiao Ai's mobile terminal 100-6, or can input the third operation for setting Xiao Ai's electronic fence in Xiao Zhong's mobile terminal 100-1 to set the electronic fence. This embodiment of the application is not limited to this.

It is understandable that an electronic fence domain is also an electronic fence, and an electronic fence domain includes multiple electronic fences and necessary routes between electronic fences. In the embodiment of the present application, the electronic fence is just a name and should not constitute a limitation to the present application. In some other embodiments of the present application, the electronic fence can also be called a safety range, etc., and the embodiment of the present application does not limit this.

In the embodiment of the present application, Xiao Ai's mobile terminal can be called a fourth electronic device, that is, the fourth electronic device includes mobile terminal 100-6.

Mobile terminal 100-6 can periodically locate Xiao Ai. When mobile terminal 100-6 detects that Xiao Ai is outside electronic fence 1 and electronic fence 2 in the first electronic fence described above, mobile terminal 100-6 can directly output a second prompt message, or send a second message to a fifth device to remind parents that Xiao Ai is currently in a dangerous area, and ask parents to contact Xiao Ai as soon as possible to ensure Xiao Ai's safety. The fifth device includes the mobile terminal of Xiao Ai's parents, other furniture and equipment in the home, and a car computer in the vehicle. Alternatively, when mobile terminal 100-6 detects that Xiao Ai is outside the second route in the first electronic fence beyond the first distance and/or beyond the first time period, mobile terminal 100-6 can directly output a second prompt message, or send a second message to the fifth device.

In other embodiments of the present application, the mobile terminal can also generate a second electronic fence. When the mobile terminal detects that the user has entered the second electronic fence, the mobile terminal can output a third prompt message to prompt the user that he has entered a dangerous area; or, the third information can be sent to terminal devices of other family members and furniture devices in the home in the same smart home account as the mobile terminal, which can trigger the terminal devices of other family members and fixed devices in the home to initiate an alarm, thereby achieving the purpose of protecting the user's personal safety.

Next, taking the example of parents setting up an electronic fence for their children, assuming that in order to prevent their children from going to dangerous places such as Internet cafes, rivers, etc., parents can set up a device for their children, such as the second electronic fence shown in Figure 8R. When the mobile terminal detects that the child enters the second electronic fence, it can trigger the mobile terminal to initiate an alarm, for example, triggering the mobile terminal to initiate an alarm to the terminal devices of other family members in the same smart home account and fixed devices in the home.

In some embodiments of the present application, in addition to controlling and using home devices through the smart home APP installed on the mobile terminal 100, family members can also directly use or control home devices. In addition, when family members directly operate home devices, the home devices will also turn on the identity authentication function to identify the currently used identity and determine whether the identity of the current user has the authority to use the home device; or, when family members directly operate home devices, the home devices will also turn on the identity authentication function and upload the collected user information corresponding to the identity authentication method to the server 300, which will identify the user's identity and determine whether the identity of the current user has the authority to use the home device. If it has the authority, the result will be sent to the home device. When the user has the authority to use the home device, it will respond normally and perform the corresponding operation; when the home device detects that the identity of the current user does not have the authority to use the home device, it will refuse to perform the corresponding operation.

Specifically, the method for the home device to identify the user's identity may be any of the following:

1. The home device sends the collected information corresponding to the user's identity authentication, such as facial information, fingerprint information, voiceprint information, operating habits, etc. to the server 300. The server 300 compares the information corresponding to the identity authentication of family members uploaded by the mobile terminal 100 with the information corresponding to the user's identity authentication uploaded by the home device, thereby identifying the user's identity and sending the identification result to the home device.

2. The home device pre-stores the identity authentication information corresponding to all family members issued by the server, and compares the above information with the collected information corresponding to the user's identity authentication to further identify the user's identity.

In other embodiments of the present application, when the home device recognizes that the current user does not have the authority to use the home device, the home device controls the mobile terminal corresponding to the current user to make instant calls, videos, etc. to the mobile terminal of the home creator or administrator based on the current user's identity. After the home creator or administrator learns of the incident, they can authorize the user in their own mobile terminal and allow the user to continue to use the home appliance normally, provided that there is no danger. In this way, it is possible to avoid situations where users force the use of the home device or even violent people destroy the home device.

The above only introduces the method of controlling home devices through mobile terminals and the method of directly controlling home appliances. However, in a smart home, there are not only application scenarios for controlling and using home devices, but also application scenarios for using mobile terminals all the time. In particular, for situations where non-owners use mobile terminals, such as children at home using their parents' mobile phones and tablets, the owners often suffer more or less financial losses, and the network environment in the parents' mobile phones will also bring unfavorable growth conditions to the children. In order to avoid this situation, the following describes how to use mobile terminals.

In an embodiment of the present application, when the mobile terminal 100, such as a mobile phone, detects that a user inputs a sensitive operation, identity authentication can be turned on. If it is identified that the current user is not the owner of the device, the event corresponding to the sensitive operation will be refused to execute, and the user will be prompted with a prompt message that the operation failed. In addition, a prompt message will be sent to other terminal devices of the owner.

Specifically, in the above process, the mobile terminal 100 can continuously perform identity authentication. That is, the mobile terminal 100 can perform periodic identity authentication within the preset time when a sensitive operation is detected. If the current user is detected as the owner of the device at any time within the preset number of times, the mobile terminal 100 executes the event corresponding to the sensitive operation; if the number of times the current user is detected as a non-owner is greater than or equal to the preset number of times, the mobile terminal 100 still refuses to execute the event corresponding to the sensitive operation. If the mobile terminal 100 detects that the user is still on the page that performs sensitive operations or using an APP that provides sensitive operations, the mobile terminal 100 will be locked.

In addition, when the mobile terminal 100 detects that someone other than the owner is performing a sensitive operation any time within the above-mentioned preset times, or when the mobile terminal 100 detects that someone other than the owner is performing a sensitive operation a preset number of times, the event will be sent to other mobile terminals equipped by the owner, such as a bracelet, to remind the owner that someone else is using the mobile terminal 100 to perform sensitive operations.

The above sensitive operations may include any one or more of the following:

1. Execute payment operations, including payment operations performed in social apps, streaming apps, payment apps, online banking apps, shopping apps, live streaming apps, etc. that provide payment functions. The amount corresponding to the payment operation can be greater than the preset value or an arbitrary value.

2. Watch game live broadcasts, videos of minors, and use game apps.

3. Delete files, text messages, phone records, chat records in social apps, and apps, etc.

Next, the above method is introduced through a specific UI implementation example.

9A-9G , which exemplarily illustrate a series of user interface diagrams after the mobile terminal 100 - 1 in the small and medium-sized enterprise detects a sensitive operation.

FIG. 9A exemplarily shows a user interface diagram provided by a social APP.

As shown in FIG9A , user interface 91 is a chat interface provided by a social APP, and user interface 91 displays a chat area 911 and a function area 912. 911 displays the chat content of Xiaozhong and Xiaojun, and the chat content shows that Xiaojun asked Xiaozhong to transfer a deposit of 520 yuan. Xiaozhong agreed and clicked the transfer icon 912A in the function area 912. In response to this operation, the mobile terminal 100 displays the transfer interface shown in FIG9B . It can be understood that Zhongjun and Xiaozhong here only indicate the accounts of two users in the social APP, and the user using the account at this time is not necessarily the account owner.

As shown in FIG. 9B , the user interface 92 is a transfer interface provided by the social APP.

The mobile terminal 100-1 can detect that the user inputs an amount of 520 as shown in FIG9B and clicks the payment control. In response to the above operation, the mobile terminal 100-1 can directly start identity authentication, or the mobile terminal 100-1 can first display the user identity authentication prompt information as shown in FIG9C. The present application embodiment does not limit this.

As shown in FIG9C , the user interface 92 displays a window 921, which displays a prompt message such as “It is detected that you are performing a payment operation, are you the owner?”, as well as controls 921A and 921B. The identity authentication method here can be the same as or different from the identity authentication method provided by the smart home APP above, or more or less, and the embodiment of the present application does not limit this.

Mobile terminal 100 - 1 may detect an operation of a user inputting a control 921A as shown in FIG. 9C , and in response to the operation, mobile terminal 100 - 1 may start identity authentication.

When the mobile terminal 100 - 1 detects that the current user is not the owner, a prompt message as shown in FIG. 9D is displayed.

As shown in FIG9D , the user interface 92 displays a window 922, as well as a control 922A and a control 922B. The window 922 displays a prompt message such as “Verification failed, payment operation is not supported temporarily.” The control 922A is used to continue identity authentication, and the control 922B is used to end identity authentication and close the payment operation interface.

Afterwards, mobile terminal 100-1 can detect the operation of the user input acting on control 922B as shown in Figure 9D. In response to the operation, mobile terminal 100-1 no longer continues identity authentication and closes the page for transferring money to Xiaojun.

Alternatively, the mobile terminal 100-1 may detect an operation of the user input acting on the control 922A shown in FIG. 9D , and in response to the operation, the mobile terminal 100-1 continues to perform identity authentication. Alternatively, the mobile terminal 100-1 may automatically perform identity authentication without detecting the operation acting on the control 922A. This embodiment of the application is not limited to this.

Optionally, when mobile terminal 100-1 continues to perform identity authentication, if the authentication fails, a prompt message is displayed again. Specifically, the prompt message may include windows 923 and 924 shown in Figures 9E and 9F to prompt the user of the number of authentication failures, the number of remaining authentication opportunities, etc.

Optionally, when mobile terminal 100-1 continues to perform identity authentication, if the verification fails and the number of failures exceeds a preset number, mobile terminal 100-1 can send a prompt message to the owner, i.e., other mobile terminals equipped by Xiaozhong. For example, a prompt message as shown in FIG9G is sent to Xiaozhong's mobile terminal 100-9, i.e., a wristband. After receiving the message, Xiaozhong can input an operation on the mobile terminal 100-9 side to agree to others to use the mobile terminal 100-1. In response to the operation, mobile terminal 100-9 can send information to mobile terminal 100-1 agreeing to others to use the machine for payment. After receiving the information, mobile terminal 100-1 can unlock and provide payment services for non-owners.

It can be understood that Figures 9A-9G above only illustrate an example of a payment operation of transferring money to others in a social software. Payment operations can also include sending red envelopes to others in the social software, or rewarding the anchor in the live broadcast room, purchasing goods, or scanning a payment code to pay others, etc. The embodiments of the present application do not limit this.

9H to 9M , which exemplarily illustrate a series of user interface diagrams after the mobile terminal 100 - 1 in the small and medium-sized enterprise detects another sensitive operation.

FIG9H exemplarily shows a schematic diagram of a scenario in which Xiao Ming uses Xiao Zhong's mobile phone to watch a live broadcast.

As shown in FIG9H , the user interface 93 is a live broadcast page provided by the live broadcast APP, and the user interface 93 displays the anchor's video area, live broadcast time, message area, and control 931, etc. The mobile terminal 100-1 can detect the operation on the control 931, and in response to the operation, the mobile terminal 100-1 expands and displays a window for giving gifts to the anchor as shown in FIG9I .

As shown in FIG9I , window 932 displays multiple gift options, such as lollipops, Mojito, small flowers, crowns, diamonds, small hearts, etc. Different gifts correspond to different prices. Assume that user Xiao Ming selects the lollipop option and clicks the gift control. In response to this operation, mobile terminal 100-1 displays the recharge window shown in FIG9J .

As shown in FIG9J , the user interface 93 displays a recharge window 933. It is understandable that the payment service provided by some APPs requires recharge, converting RMB into a payment method supported by the APP, such as "Y Coins", after which the user can use the payment method supported by the APP to make payment operations.

The mobile terminal 100-1 can detect that the user inputs a recharge value of 520 as shown in FIG9J , and in response to the operation, the mobile terminal 100-1 can directly start identity authentication, or the mobile terminal 100-1 can first display the user identity authentication prompt information as shown in FIG9K . The present application embodiment does not limit this.

As shown in FIG9K , the user interface 93 displays a window 934, which displays a prompt message such as “It is detected that you are performing a payment operation, are you the owner?”, as well as controls 934A and 934B. The identity authentication method here can be the same as or different from the identity authentication method provided by the smart home APP above, or more or less, and the embodiment of the present application does not limit this.

Mobile terminal 100 - 1 may detect an operation of the user input acting on control 934A as shown in FIG. 9K , and in response to the operation, mobile terminal 100 - 1 may start identity authentication.

When the mobile terminal 100 - 1 detects that the current user is not the owner, it displays a prompt message as shown in FIG. 9L .

As shown in Figure 9L, the user interface 93 displays a window 935, as well as controls 935A and 935B. Window 935 displays a prompt message such as "Verification failed, recharge operation is not supported yet". Afterwards, the mobile terminal 100-1 can detect the user input acting on the operation of the control 935B shown in Figure 9D. In response to the operation, the mobile terminal 100-1 no longer continues to perform identity authentication and closes the recharge page. Alternatively, the mobile terminal 100-1 can detect that the user input acts on the control 935A and the operation shown in Figure 9D. In response to the operation, the mobile terminal 100-1 continues to perform identity authentication. Alternatively, the mobile terminal 100-1 does not need to detect the operation acting on the control 922A, and can automatically perform identity authentication. The embodiments of the present application are not limited to this.

When the mobile terminal 100 - 1 detects that the user inputs an operation on the control 935B as shown in FIG. 9L , in response to the operation, the mobile terminal 100 - 1 continues to display the live broadcast screen as shown in FIG. 9M .

It will be understood that the sensitive operations described in the present application are not limited to the two payment operation examples shown in FIGS. 9A-9G and 9H-9M above. There are other sensitive operations. For example, when it is detected that a user is playing online gambling games or purchasing goods, scanning codes to pay, etc., the mobile terminal 100 will also implement the above method to prevent the user from continuing to perform sensitive operations or performing sensitive behaviors, etc. The embodiments of the present application will not go into details about this.

Next, referring to FIG. 9N-FIG . 9Q , FIG. 9N-FIG . 9Q exemplarily illustrate a series of user interface schematic diagrams after the mobile terminal 100 - 1 in the small and medium-sized enterprise detects another sensitive operation.

FIG9N exemplarily shows a schematic diagram of a scenario in which other users use Xiaozhong’s mobile terminal to scan a code to pay.

As shown in Fig. 9N, the user interface 94 is a barcode scanning interface, which displays an image acquisition area 941, which displays an image acquired by the mobile terminal 100-1, such as Xiaojun's payment code. When the mobile terminal 100-1 successfully scans Xiaojun's payment code, the payment interface shown in Fig. 9O can be displayed.

As shown in FIG. 9O , user interface 95 is a payment interface, which can support users to pay Xiaojun.

As shown in FIG. 9P , mobile terminal 100 - 9 can detect that the user inputs 200 yuan and acts on the payment control. In response to this operation, mobile terminal 100 - 1 can first display the user identity authentication prompt information as shown in FIG. 9Q .

As shown in FIG9Q , the user interface 95 displays a window 952, which displays a prompt message such as "It is detected that you are performing a payment operation, are you the owner of the device?", as well as controls 952A and 952B. Among them, control 952A is used to turn on the identity authentication function, and control 952B is used to close window 952 and refuse to pay 200 yuan to Xiaojun. The identity authentication method here can be the same as or different from the identity authentication method provided by the smart home APP above, or more or less, and the embodiment of the present application does not limit this.

When the mobile terminal 100-1 detects the operation of the control 952A shown in FIG9Q, in response to the operation, the mobile terminal 100-1 starts identity authentication. When the identity authentication is successful, that is, when the current user is the owner of the mobile terminal 100-1, Xiaozhong, the mobile terminal will pay Xiaojun; and when the identity authentication fails, the user interface displayed by the mobile terminal 100-1 can refer to the description of FIG9D-FIG9G above, which will not be repeated here.

In other embodiments of the present application, when the mobile terminal 100-1 detects a payment operation, the identity authentication function will be turned on when the payment amount is greater than a preset amount, such as 100 yuan, and when the payment amount is less than or equal to the preset amount, the identity authentication function will not be turned on, and the payment will be made directly to the other party. Alternatively, when the mobile terminal 100-1 detects a payment operation, if the payment object is an infrequent contact, the identity authentication function will be turned on, and when the payment object is a frequent contact, the identity authentication will not be turned on, and the payment will be made directly to the other party.

In other embodiments of the present application, the mobile terminal 100-1 may not display the prompt information shown in Figure 9Q in response to the payment operation shown in Figure 9P, and may directly start identity authentication, which is not limited by the embodiments of the present application.

9R-9W , which exemplarily illustrate a series of user interface diagrams after the mobile terminal 100 - 1 in the small example detects another sensitive operation.

When the mobile terminal detects that a user is watching live broadcasts, underage videos, or playing games, identity authentication can be enabled. If a user other than the owner is detected, the user is prohibited from watching live broadcasts, underage videos, or playing games, etc. Next, an example is given of Xiao Ai using Xiao Zhong's mobile terminal 100-1 to watch live broadcasts.

As shown in FIG. 9R , the user interface 96 displays multiple options corresponding to live broadcast rooms, and the user can click on any one of the options to enter the corresponding live broadcast room and watch the live broadcast.

When the mobile terminal 100 - 1 detects an operation on option 961 , in response to the operation, the mobile terminal 100 - 1 directly starts identity authentication, or first displays a prompt message for starting identity authentication as shown in FIG.

As shown in FIG9S , the user interface 96 displays a window 962, which displays a prompt message such as “It is detected that you are performing a payment operation, are you the owner of the device?”, as well as controls 962A and 962B. Among them, control 962A is used to turn on the identity authentication function, and control 962B is used to close window 962 and refuse to enter the live broadcast room. The identity authentication method here can be the same as or different from the identity authentication method provided by the smart home APP above, or more or less, and the embodiment of the present application does not limit this.

When the mobile terminal 100-1 detects the operation of the control 952A shown in FIG9Q, in response to the operation, the mobile terminal 100-1 starts identity authentication. After the mobile terminal starts identity authentication, if the authentication is successful, it enters the live broadcast room corresponding to option 961 and displays the live broadcast content; if the authentication fails, the mobile terminal will display the authentication failure prompt information shown in FIG9T.

As shown in FIG9T , the user interface 96 displays a window 963, which displays a prompt message such as “Verification failed, live broadcast viewing is not supported temporarily”, as well as controls 963A and 963B. Control 963A is used to continue identity authentication, and control 963B is used to end identity authentication. In some embodiments of the present application, control 963A may not be displayed in window 963.

Afterwards, mobile terminal 100-1 can detect the operation of user input acting on control 963B as shown in Figure 9T. In response to the operation, mobile terminal 100-1 no longer continues identity authentication and refuses the user to enter the live broadcast room to display the live content.

Alternatively, the mobile terminal 100-1 may detect an operation of the user input acting on the control 963A shown in FIG9T , and in response to the operation, the mobile terminal 100-1 continues to perform identity authentication. Alternatively, the mobile terminal 100-1 does not need to detect an operation acting on the control 963A, and when the display time of the window 963 exceeds a preset time, the mobile terminal 100-1 may automatically perform identity authentication. The embodiments of the present application are not limited to this.

Optionally, when the mobile terminal 100-1 continues to perform identity authentication, if the authentication fails, the prompt information shown in FIG. 9T is displayed again; if the authentication fails, and the number of failures exceeds a preset number, for example, 3 times, the mobile terminal 100-1 may take any one or more measures:

The mobile terminal 100 - 1 displays the prompt information as shown in FIG. 9U , and then locks the screen, that is, displays the lock screen interface as shown in FIG. 9V ; or the mobile terminal only stops running the APP that provides the live broadcast.

Or the mobile terminal 100-1 can send a prompt message to the owner himself, that is, other mobile terminals equipped by Xiaozhong, for example, to the mobile terminal 100-9, that is, a wristband or a car computer of Xiaozhong.

When Xiaozhong is driving the vehicle, the vehicle-mounted computer can receive the prompt message sent by the mobile terminal 100-1 and output the prompt information. The prompt information can be output in the form of voice broadcast or displayed in the user interface shown in FIG9W.

As shown in FIG9W , the vehicle computer displays a prompt message "Others want to use your mobile terminal 100-1 to watch live broadcasts", as well as an approval control and a rejection control. Xiaozhong can input an operation on the vehicle computer end to agree that others use the mobile terminal 100-1 to watch live broadcasts. In response to this operation, the vehicle computer can send a message to the mobile terminal 100-1 agreeing that others use the machine to watch live broadcasts. After receiving this message, the mobile terminal 100-1 can unlock the screen or unlock the APP that provides live broadcasts, and provide live broadcast viewing services for non-machine owners.

The following only introduces how parents can achieve their children's educational goals by setting rules for their children's use of electronic devices. However, in family education, it is not only necessary to achieve the purpose of education by disciplining children's learning and living habits, but also education requires parents to teach by words and deeds, because parents are the best enlightenment teachers for their children.

Therefore, in order to achieve a comprehensive education for children and to achieve the purpose of parents spending high-quality time with their children, the control method of home devices provided by this application also includes setting rules for parents to use electronic devices. The details are as follows:

In an embodiment of the present application, the mobile terminal 100 may set rules for parents to use home devices, which may be the default settings of the mobile terminal 100 or may be user-defined. The rules for parents to use home devices specifically refer to: when the child's status is a specified status, the home device does not support parents to perform specified operations. The above-mentioned specified status includes any one or more of the following: the child is not asleep, the child is studying, or the child is not in the same room with the parents. The embodiment of the present application does not limit the specified status of the child. The above-mentioned specified operations include any one or more of the following: playing games, watching TV series, live broadcasts, short videos, etc. The embodiment of the present application does not limit the specified operations that parents are restricted from.

In the embodiment of the present application, the device for detecting the status of the child can be any one or more of the following in the home device 200: monitoring devices such as cameras, portable terminal devices such as mobile phones, tablets, notebooks, smart watches, sports bracelets, etc. The embodiment of the present application does not limit this. The above-mentioned device for detecting the status of the child can also be called the sixth device.

In the embodiment of the present application, the device used to restrict parents from performing designated operations may be any one or more of the following in the home device 200: a mobile phone, a tablet, a notebook, a television, a projection device, a game console, etc. The embodiment of the present application does not limit this.

Next, we will continue to use the father and daughter, Xiaoai and Xiaozhong, to introduce how to control home devices.

Scenario 1: Xiaozhong is at home accompanying Xiaoai. Before this, in order to be a good father and set a good example for Xiaoai, Xiaozhong can use the mobile terminal 100-1 to set rules for using the home device 200. The rule is, for example, that when Xiaoai is not sleeping, studying, or Xiaoai and her father are in the same room, her father is not allowed to use the home device 200 to play games, watch TV series, or watch live broadcasts.

FIG9X shows an example of a schematic diagram of an operation interface for watching a live broadcast using the mobile terminal 100-1 in a small group.

As shown in Fig. 9X, the user interface 121 is the desktop of the mobile terminal 100-1. The desktop displays icons corresponding to live broadcast applications, icons corresponding to game applications, icons corresponding to short video applications, and icons corresponding to Huawei Video, etc.

Assuming that Xiaozhong wants to open a live broadcast application to watch live broadcast at this time, mobile terminal 100-1 can detect the operation acting on the live broadcast application icon shown in Figure 9X. In response to this operation, mobile terminal 100-1 determines whether Xiaozhong can watch the live broadcast based on the acquired status of Xiaoai.

Among them, the mobile terminal 100-1 obtains Xiao Ai's status by any one or more of the following methods.

In the first method, the mobile terminal 100-1 responds to the detected operation on the icon of the live broadcast application, and in response to the operation, triggers the mobile terminal 100-1 to control other home devices 200 to detect the status of Xiao Ai. For example, the home monitoring device such as a camera is controlled to obtain the status of Xiao Ai, or the home device 200 such as Xiao Ai's smart watch, bracelet, and mobile phone is controlled to obtain the status of Xiao Ai.

Second, the mobile terminal 100-1 can periodically receive the status of Xiao Ai detected by other home devices 200 and store data related to the status of Xiao Ai. The mobile terminal 100-1 responds to the operation detected on the icon of the live broadcast application. In response to the operation, the mobile terminal 100-1 retrieves the relevant data of the most recent status received by Xiao Ai to determine whether Xiao Ai can watch the live broadcast.

Thirdly, the mobile terminal 100 - 1 obtains the child's status according to the operation related to the child's status input by the user.

In the embodiment of the present application, the above-mentioned operation acting on the icon of the live broadcast application can also be called the seventh operation, and the seventh operation is used to open the live broadcast application of the mobile terminal 100-1, that is, to activate the second function of the first device.

Assume that the state of Xiao Ai obtained by mobile terminal 100-1 is not asleep yet. At this time, mobile terminal 100-1 will output a prompt message prohibiting the user from watching the live broadcast. The prompt message can be output in the form of a language broadcast, or it can be displayed as a prompt message as shown in Figure 9Y. This embodiment of the present application does not limit this.

As shown in FIG. 9Y , the mobile terminal 100 - 1 displays a prompt message such as “It is detected that Xiao Ai has not slept yet, you cannot watch the live broadcast yet” and refuses to run the live broadcast application.

It is understandable that the user interface diagram of the mobile terminal 100-1 shown in Figures 9X-9Y is only an example when Xiao Ai is in a specified state according to the specified operation performed by Xiao Zhong, and Xiao Ai is in a specified state, and should not constitute a limitation on the present application. In some other embodiments, the specified operation detected by the mobile terminal 100-1 also includes an operation for opening a game application or a short video application, and the operation is not limited to the operation acting on the icon of the application, but can also be in the form of inputting a voice command to open the application, etc. In some other embodiments, the mobile terminal 100-1 detects that the user is in a specified state, in addition to not sleeping, and also includes, for example, studying or being in the same room with parents, etc. In some other embodiments, the prompt information output by the mobile terminal can also be in the form of a text message reminder, or a window reminder, or it can be displayed in a module for receiving messages in the smart home APP, or a voice reminder, etc., and the embodiments of the present application are not limited to this.

In other embodiments of the present application, when it is detected that the parents are sleeping or away from home, when the child uses the parents' electronic device, Xiao Ai will be prohibited from using the parents' electronic device, or Xiao Ai will be prohibited from using some functions of the parents' electronic device, etc. The details are as follows:

In an embodiment of the present application, the mobile terminal 100 can set rules for children to use home devices, which can be the default setting of the mobile terminal 100 or customized by the user. The rules for parents to use home devices specifically refer to: when the status of the parents is a specified status, the home device does not support the parents to perform specified operations. The above-mentioned specified status includes any one or more of the following: parents leaving home, parents sleeping, etc. The embodiment of the present application does not limit the specified status of parents. The above-mentioned specified operation can be called the eighth operation, and the eighth operation is used to activate the third function of the mobile terminal 100, that is, your first device, and the third function includes any one or more of the following: unlocking the screen, starting to play games, watching TV series, etc. The embodiment of the present application does not limit the specified operations that parents are restricted from.

In the embodiment of the present application, the device for detecting the parent's status may be any one or more of the following in the home device 200: monitoring devices such as cameras, portable terminal devices such as mobile phones, tablets, smart watches, sports bracelets, etc. The embodiment of the present application does not limit this.

In the embodiment of the present application, the device for restricting the child from performing the specified operation can be any one or more of the following in the home device 200: a mobile phone, a tablet, a notebook, a television, a projection device, a game console, etc. The embodiment of the present application does not limit this. The above-mentioned device for detecting the status of the parents can also be called the sixth device.

Next, we will continue to use the father and daughter, Xiaoai and Xiaozhong, to introduce how to control home devices.

When the mobile terminal 100-1 detects that a user is using an electronic device, it can identify the user's identity. For example, when the current user is identified as Xiao Ai, the mobile terminal 100-1 can obtain the status of Xiao Ai's parents, Xiao Zhong and/or Xiao Hua. For example, the status obtained is away from home or sleeping. The mobile terminal 100-1 determines whether to allow Xiao Ai to use the mobile terminal 100-1 or allow Xiao Ai to use some functions of the mobile terminal, etc., according to the user's pre-defined usage rules or the default rules set by the mobile terminal 100-1. The user interface schematic diagram of Xiao Ai using Xiao Zhong's mobile terminal 100-1 to perform a specified operation when Xiao Zhong is in a specified state is similar to the user interface schematic diagram of Xiao Zhong performing a specified operation when Xiao Ai is in a specified state, and will not be repeated here.

In a smart home, in addition to kitchen appliances, cleaning appliances, and sports equipment, home devices also include many electronic devices for education and learning. These home devices are important elements for building a smart home. In family life, it is indispensable that parents need to educate their children at home, so that they can cultivate good living habits, learning habits, etc. In the process of family education, education is inseparable from learning rules and living habits, but parents usually cannot always supervise their children's learning and life.

In order to save parents' time and not delay their children's education, this application provides a method for controlling home devices to achieve smart home education. Users can set the use rules of home devices according to their children's education plan. If the children break the rules, they must complete the tasks set in the corresponding home devices to achieve the education of their children. This means creating the concept of smart tutoring in a smart home.

Next, the control method of the above-mentioned home device is introduced in combination with the UI implementation example, still taking the two protagonists, the father Xiaozhong and the daughter Xiaoai, and their equipped mobile terminals as examples.

In the embodiment of the present application, Dad Xiaozhong can use his own mobile phone, such as mobile terminal 100-1, to set the rules for his daughter Xiaoai to use the home devices, or directly set the rules for Xiaoai to use in the corresponding home devices, and the embodiment of the present application does not limit this. However, considering that Dad Xiaozhong is the creator of the family, it is more reasonable and convenient to concentrate the management, setting, and control of the home devices in Dad Xiaozhong's mobile terminal 100-1 for processing. Therefore, the following UI embodiment only takes the setting of usage rules in mobile terminal 100-1 as an example for introduction.

In addition, the home device for which the above-mentioned usage rules are set may specifically be a tablet of my daughter Xiao Ai, such as mobile terminal 100-6, or other devices in the home, such as computers, smart screens, projectors, and the like. The embodiments of the present application do not limit this. However, considering the frequency with which my daughter Xiao Ai uses home devices, mobile terminal 100-6 is an intelligent terminal that integrates learning, entertainment, communication, etc., and its frequency of use is the highest among all home devices. Therefore, the following UI embodiment only takes mobile terminal 100-6 as an example of the setting object of the usage rules for introduction.

In addition, the above tasks may specifically include any one or more of the following: tasks such as learning and exercising completed by daughter Xiao Ai using the mobile terminal 100-6 Zhang Gong, the object of the rule setting; using other household equipment such as sports equipment treadmills and bracelets to complete tasks such as running and fitness; using other household equipment such as learning equipment computers and learning machines to complete tasks such as doing exercises, reading, and attending classes; or using other household equipment such as furniture cleaning equipment vacuum cleaners to complete the task of cleaning the room. The embodiments of the present application do not limit this.

In a specific implementation, Xiao Zhong can bind Xiao Ai's mobile terminal 100-6 in the mobile terminal 100-1, and set the usage rules of the mobile terminal 100-6 according to the education plan for Xiao Ai. In addition, Xiao Zhong also needs to formulate educational measures to be taken after breaking the above-mentioned usage rules, and the educational measures are to formulate one or more tasks, which can be completed by Xiao Ai's mobile terminal 100-6 or other home devices at home. For example, the usage rules set by Xiao Zhong are that the duration of using the mobile terminal 100-6 every day is 4 hours, and the tasks to be completed after breaking the rules are, for example, learning for 1 hour and exercising for 1 hour. When Xiao Ai uses the mobile terminal 100-6 for more than 4 hours, the mobile terminal 100-6 can lock the screen or lock some functions until the mobile terminal 100-6 detects that Xiao Ai uses the mobile terminal 100-6 to complete the above tasks; or the mobile terminal 100-6 receives the relevant data of Xiao Ai using other home devices such as a treadmill to complete a 1-hour exercise task, and receives the relevant data of Xiao Ai using other home devices such as a learning machine to complete a 1-hour learning task, the mobile terminal 100-6 will be unlocked, or the lock of some functions will be released.

10A-10H , which exemplarily illustrate a series of operation interface diagrams for Xiao Zhong to set usage rules and related tasks of the device in the mobile terminal 100 - 1 .

As shown in FIG. 10A , the user interface displayed by the mobile terminal 100 - 1 is similar to the user interface 61 shown in FIGS. 6K , 8F , and 8J above. The description of FIG. 6K in the above text is also applicable to the description of FIG. 10A , which will not be repeated here.

The mobile terminal 100 - 1 detects an operation of the user clicking the control 615D, and in response to the operation, the mobile terminal 100 - 1 displays the user interface 81 as shown in FIG. 10B .

As shown in FIG. 10B , the user interface displayed by the mobile terminal 100 - 1 is similar to the user interface 81 shown in FIGS. 6K , 8F , and 8J above. The description of FIG. 6K in the above text is also applicable to the description of FIG. 10A , which will not be repeated here.

Mobile terminal 100-1 can detect the user clicking on option 814 in Figure 10B. Option 814 is used to open my other devices page, which can specifically provide the user with the function of managing the user's other devices such as tablets, notebooks, bracelets, headphones, etc. Therefore, in response to the above operation, mobile terminal 100-1 displays the user interface 101 as shown in Figure 10C.

As shown in FIG10C , the user interface 101 displays options 1011, 1012, 1013, etc. corresponding to the bound electronic devices, as well as a control 1014 for binding more devices. Option 1011 corresponds to the mobile terminal 100-9 equipped with Xiao Zhong, option 1012 corresponds to Xiao Ai's mobile terminal 100-5 such as a smart watch, and option 1013 corresponds to Xiao Ai's mobile terminal 100-6 such as a tablet computer. The user can set the usage rules of the corresponding device by clicking on the above options, as follows:

Mobile terminal 100-1 can detect an operation on any device option in FIG10C, and in response to the operation, display the usage rule setting page corresponding to the device. For example, mobile terminal 100-1 can detect an operation of a user clicking option 1013 in FIG10C, and in response to the operation, mobile terminal 100-1 displays the usage rule setting page of mobile terminal 100-6 as shown in FIG10D.

As shown in FIG10D , the user interface 102 is a page for setting the usage rules of the mobile terminal 100 - 6. A series of default rule options are displayed in the user interface 102, such as option 1021 for setting the usage duration of the mobile terminal 100 - 6 in one day, option 1022 for setting the usage duration of the mobile terminal 100 - 6 once, option 1023 for setting the usage time period of the mobile terminal 100 - 6, option 1024 for setting prohibited APPs, etc. The user can set the usage duration, usage time period, prohibited APPs, etc. by clicking on the above options.

The user interface 102 also displays default options for unlocking the mobile terminal 100-6 after breaking the above usage rules, such as option 1025 for setting exercise time, option 1026 for setting study time, and option 1027 for setting password authentication, etc. The user can set the study time, exercise time period, and password for unlocking, etc. by clicking on the default options given by the mobile terminal 100-1.

In addition to the above default setting options, the user interface 102 also displays a control 1028A and a control 1028B, wherein the control 1028A is used by the user to re-customize the usage rules and unlocking tasks, and the control 1028B is used to apply the above default options that have been set. In other words, the user can choose to set the usage rules and unlocking task options given by default by the mobile terminal 100-6, and then click the control 1028B to apply the already set usage rules and restriction methods. Or the user can customize the usage rules and unlocking tasks of the mobile terminal 100-6 by clicking the control 1028A.

It is understandable that the usage rules shown in FIG. 10D are only an example. In other embodiments of the present application, the usage rules displayed by the mobile terminal 100-1 may include more or fewer options than those shown in FIG. 10D. For example, it may also include options for APPs that are allowed to be used instead of options for APPs that are restricted to be used, and options for the duration of the application that is allowed to be used continuously, etc. The user can click on the option to set the APPs that are allowed to be used, such as educational and learning APPs and communication APPs such as text messages and calls. When the mobile terminal 100-1 detects that the user wants to use other APPs outside the scope of the allowed APPs, such as game videos or video APPs, the mobile terminal 100-1 may directly refuse to run the APP, or the mobile terminal 100-1 may prompt the user that the specified unlocking task must be completed before the APP can be used. When the mobile terminal detects that the user has completed the specified unlocking task, the user is allowed to use the APP.

In the embodiment of the present application, the application program allowed to be used included in the first rule may also be referred to as the first application program. The duration of the allowed continuous use of the first application program is the third duration.

In other embodiments of the present application, the usage rules can not only set the APPs that are allowed to be used, but can also further set that only specified functions, specified businesses, specified services, etc. in the APP are allowed to be used. Taking video APPs as an example, it can be set that video APPs can only play educational and learning videos, while videos other than educational and learning videos, such as games and entertainment, are prohibited from playing. When the mobile terminal 100-1 detects that the user breaks the usage rules, such as watching a non-learning APP in Huawei Video, in response to the operation, the mobile terminal 100-1 can jump to the educational and learning APP provided by Huawei Video by default. When the user completes watching and learning the educational and learning video, he can continue to watch the non-educational and learning video. Alternatively, after the mobile terminal 100-1 jumps to the educational and learning video provided by the Huawei Video APP, the user can exit the page and watch the non-educational and learning video by completing other unlocking tasks.

As shown in FIG10E , the mobile terminal 100-1 can detect the operation for setting the usage rules of the mobile terminal 100-1 and the unlocking task. For example, the user sets the above default options to use the mobile terminal 100-6 for no more than the first duration, such as 4 hours a day, and the continuous use time is no more than the second duration, such as 1 hour, and the use time period is the third time period, such as 8:30-12:00 in the morning and 14:00-18:00 in the afternoon, etc., and the restricted APPs are Live A and Reading A, etc. For example, the user sets the above default options for unlocking to 1 hour of activity time, 1 hour of study time, or to enable the unlocking password, etc.

As shown in Figure 10F, when the mobile terminal 100-1 detects an operation that acts on the control 1028B, and the mobile terminal 100-6 detects that the user has broken any one or more of the rules, the mobile terminal 100-6 will lock or restrict some functions. If the user wants to unlock the mobile terminal 100-6, or release some restricted functions in the mobile terminal 100-6, it is necessary to complete the set unlocking task. Specifically, when the mobile terminal 100-6 detects that any one or more of the above unlocking tasks meet the conditions, the mobile terminal 100-6 can be unlocked, or the restricted functions in the mobile terminal 100-6 can be released.

In some other embodiments of the present application, the mobile terminal 100-1 may also detect an operation input by a user for customizing the usage rules and unlocking tasks of the mobile terminal 100-1. For example, the user may click on the control 1028A shown in FIG. 10G, and in response to the operation, the mobile terminal 100-6 displays the custom page shown in FIG. 10H.

FIG. 10H exemplarily shows a user interface diagram for customizing usage rules and unlocking tasks.

As shown in FIG10H , the user interface 103 displays a customized usage rule 1031 and a control 1032 for adding usage rules. There are also two customized unlocking tasks 1033 and 1034, and a control 1035 for adding unlocking tasks. Among them, the customized usage rule 1031 is specifically: set the type of APP used to be a game APP, set the total time of using the game APP in a day to 3 hours, set the total time of using the APP once to 1 hour, set the time period for using the game APP to 8:30-12:00 in the morning and 14:00-18:00 in the afternoon, etc. Among them, the customized unlocking task 1 is specifically: study for 1 hour and run for 1 hour; the customized unlocking task 2 is specifically: Xiaozhong's fingerprint, Xiaohua's fingerprint, etc.

It is understandable that FIG. 10H is only a schematic diagram of a custom page, in other embodiments of the present application. The relevant content of the use rules that the user can customize may also include, for example: the total time of using a certain APP or a certain type of APP in one day, the time of using a certain APP or a certain type of APP each time, or the time of using a certain or a certain type of APP, or prohibiting the use of a certain or a certain type of APP, or prohibiting the use of a certain or a certain type of APP. The specified function in the APP, etc. In addition, in other embodiments of the present application, the option of the above-mentioned use time period not only sets a specific time, but also sets the use time period by date. For example, for children, they need to go to school from Monday to Friday, and Saturday and Sunday are rest days. Therefore, the use time period can also be personalized according to the date, for example, from Monday to Friday, Xiao Ai is not allowed to play electronic devices, or can only play for a while at 18:00-19:00 in the evening, and on Saturday and Sunday, you can play electronic devices without affecting the rest. For example, it can be set to 8:30-12:00 in the morning and 14:00-18:00 in the afternoon every Saturday and Sunday, and the embodiment of the present application does not limit the setting of the use time. The related contents of the unlocking task that can be customized by the user may include, for example: any one or more of the multiple designated users input the command for unlocking to the mobile terminal 100-6, such as entering a password, or fingerprint, or facial information, or sending a verification code to the mobile terminal 100-6; or the tasks that Xiao Ai must perform can also be set, which can be sports, learning, or housework, and the platform for completing these tasks can be completed in Xiao Ai's native mobile terminal 100-6, or in other devices in the home device. For example, Xiao Ai can complete the task of studying for 1 hour in the mobile terminal 100-6, or Xiao Ai can complete the task of running for 1 hour on the treadmill, or Xiao Ai uses household cleaning equipment such as a vacuum cleaner to clean the whole house, etc. The embodiment of the present application does not limit the setting of usage rules and unlocking tasks.

In the embodiment of the present application, the usage rules of the mobile terminal 100-6 exemplarily shown in Figures 10F and 10H above can be referred to as the first rule, and the unlocking tasks exemplarily shown in Figures 10F and 10H above can be referred to as the first task. The operation for setting the first rule shown in Figures 10C-10H can be referred to as the fourth operation, and the user can set the first rule of the mobile terminal 100-6, i.e., the fourth device, by entering the fourth operation in the mobile terminal 100-1, i.e., the first device, and then the mobile terminal 100-1 can send the first rule to the mobile terminal 100-6. Alternatively, the mobile terminal 100-6 can directly receive the fourth operation input by the user to set its own first rule.

Assuming that mobile terminal 100-1 applies the customized usage rules and unlocking tasks shown in Figures 10G-10H above, the following introduces the operating interface diagram of mobile terminal 100-6 when the usage rules and unlocking tasks set in mobile terminal 100-1 are used.

11A-11R , which exemplarily show a series of operation interface diagrams of the mobile terminal 100 - 6 when being used by Xiao Ai based on the usage rules and unlocking tasks set by Xiao Zhong in the mobile terminal 100 - 1 .

First, FIG. 11A to FIG. 11E exemplarily show a series of user interface schematic diagrams when the mobile terminal 100 - 6 detects that Xiao Ai violates one of the usage rules.

FIG. 11A is a schematic diagram showing interfaces of a series of application programs installed in the mobile terminal 100 - 6 .

As shown in FIG. 11A , the user interface 111 , i.e., the desktop, displays icons of multiple applications, including an application icon 1111 of Game A.

The mobile terminal 100-6 can detect an operation for opening a game, such as an operation on the application icon 1111 of the game A shown in FIG. 11A . In response to the operation, the mobile terminal 100-6 will refuse to run the game A according to one of the rules of the usage rules set in FIG. 10H above, "the time period for using the game APP is 8:30-12:00 and 14:00-18:00". In one implementation, the mobile terminal 100-6 can also display a prompt message as shown in FIG. 11B .

As shown in FIG11B , the user interface 111 displays a window 1112. The window 1112 displays information for reminding the user of the rules for playing the game, such as a prompt message that “Now is not the game time, you can play the game between 8:30-12:00 and 14:00-18:00.” The window 1112 also displays a control 1112A and a control 1112B, wherein the control 1112A is used to unlock the game and obtain the right to play the game; and the control 1112B is used to close the window 1112.

In one possible case, when the mobile terminal 100-6 displays the user interface shown in FIG. 11B, if Xiao Ai wants to continue playing the game, she can click on the control 1112A. In response to the operation, the mobile terminal 100-6 will display the unlocking task prompt information as shown in FIG. 11C.

As shown in FIG11C , the user interface 111 displays a window 1113. Window 1113 displays a method for prompting the user to select an unlocking game, and options corresponding to the unlocking task, such as option 1113A and option 1113B. Option 1113A and option 1113B are specifically as follows: the mobile terminal 100-6 displays option 1113A corresponding to tasks such as studying for 1 hour and running for 1 hour, and displays option 1113B corresponding to the fingerprint of Xiaozhong and the fingerprint of Xiaohua, according to the unlocking task of the mobile terminal 100-6 set by Xiaozhong in the mobile terminal 100-1 shown in FIG10H above.

At this point, Xiao Ai can unlock the game by clicking on any of the above options and using the corresponding unlocking method. The unlocking task will not be described in detail here, and will be described in detail in Figures 11M-11R below.

In another possible case, when the mobile terminal 100-6 displays the user interface shown in FIG. 11D (FIG. 11B), if Xiao Ai does not want to play the game, she can click on the control 1112B, and in response to the operation, the mobile terminal 100-6 will stop displaying the window 1112 as shown in FIG. 11E. After that, Xiao Ai can wait until after 8:30 to open the game.

In the embodiment of the present application, the prompt information for executing the unlocking task exemplarily shown in FIG. 11B-FIG 11C may also be referred to as fourth prompt information.

Next, FIGS. 11F-11L exemplarily show a series of user interface schematic diagrams when the mobile terminal 100 - 6 detects that Xiao Ai has violated another usage rule.

As shown in Fig. 11F, the time displayed by mobile terminal 100-6 is 8:30, which is the time set by Xiaozhong to play games. Therefore, at this time, mobile terminal 100-6 can detect the operation on icon 1111, and in response to the operation, mobile terminal 100-6 will display the user interface schematic diagram provided by game A shown in Fig. 11G.

As shown in FIG11G , the user interface 112 is the homepage provided by Game A. Xiao Ai can click to start the game and experience the game. The mobile terminal 100-6 can start recording the time of playing the game from the time when the operation acting on the start game control is detected, or can start recording the time of playing the game from the time when the operation for opening the game A application is detected. This embodiment of the present application will not be described in detail. The following embodiment will take 8:30 as an example to start recording the time of Xiao Ai playing the game.

As shown in FIG11H , user interface 113 provides a game interface for game A. When mobile terminal 100-6 detects that game A has been running for a certain time, it will display a prompt message to prompt the user how long the game has been played and how much time is left. For example, if the mobile terminal detects that Xiao Ai plays the game from 8:30 to 9:10, mobile terminal 100-6 will display prompt message 1131. Prompt message 1131 displays a prompt message that "It is detected that your game time has reached 40 minutes. If you play for another 20 minutes, your current game time will be full."

After the mobile terminal 100-6 displays the prompt information as shown in FIG11H, Xiao Ai can choose to stop the game, exit the game, or continue playing the game. Assuming that Xiao Ai continues the game, when Xiao Ai finishes the current game, the mobile terminal 100-6 will display the user interface as shown in FIG11I.

As shown in Figure 11I, the user interface 114 is a mid-game interface after a game is finished. The time is 9:20 and there are 10 minutes left according to the prescribed game time. If the mobile terminal 100-6 detects that Xiao Ai operates the control shown in Figure 11I for starting the next game, the mobile terminal 100-6 can display a prompt message as shown in Figure 11J.

As shown in FIG11J , the user interface 114 displays a window 1141, which displays a prompt message "It is detected that your game time is only 10 minutes left. Start a new game now, which may interrupt the game", and also displays controls 1141A and 1141B. Among them, control 1141A is used to continue to start a new game, and control 1141B is used to exit the game. Assuming that Xiao Ai wants to continue a new game, the mobile terminal 100-6 can detect the operation acting on 1141A, and in response to the operation, the mobile terminal 100-6 will start a new game.

When the mobile terminal 100-6 detects that Xiao Ai is playing a new game and the game time has reached 1 hour, that is, when the mobile terminal 100-6 detects that the time is 9:30, the mobile terminal 100-6 will prompt the user that the game time is full and force the user to exit the game; or, the mobile terminal 100-6 will prompt the user that the game time is full, but will not force the user to exit the game until the game is finished, and then the mobile terminal 100-6 will exit the game, that is, stop running the game APP. This embodiment of the application is not limited to this.

Next, a schematic diagram of a user interface displayed by the mobile terminal 100 - 6 is shown using the first possibility, that is, the mobile terminal 100 - 6 displays a prompt message indicating that the game time is full and forces the user to exit the game.

As shown in FIG11K , user interface 115 provides a game interface for game A. When mobile terminal 100-6 detects that the specified time has been reached for running game A, it will display a prompt message to remind the user that the game time is up and the game is about to be exited. For example, if the mobile terminal detects that Xiao Ai starts playing the game from 8:30 to 9:30, the mobile terminal 100-6 will display prompt message 1151. Prompt message 1151 displays a prompt message that "It is detected that your game time has reached 1 hour. The game time has been used up and you are about to exit the game."

Afterwards, the mobile terminal 100 - 6 will exit the game being played and display the home page provided by the game A, or stop running the game A and display the desktop. The embodiment of the present application does not limit this.

As shown in FIG. 11L , FIG. 11L exemplarily shows that the mobile terminal 100-6 detects that the game time is full, exits the game being played and displays the home page provided by game A. At this time, the home page displays a window 1121, and the window 1121 displays "The current game time has reached 1 hour. If you want to continue the game, you need to unlock the game." In addition, the window 1121 also displays a control 1121A and a control 1121B. Among them, the control 1121A is used to unlock the game, and the control 1121B is used to exit the game.

In one possible case, when the mobile terminal 100-6 displays the user interface shown in FIG. 11L, if Xiao Ai wants to continue playing the game, she can click on the control 1121A. In response to the operation, the mobile terminal 100-6 will display the unlocking task prompt information as shown in FIG. 11M.

In another possible case, when the mobile terminal 100-6 displays the user interface shown in FIG. 11L, if Xiao Ai does not want to play the game, she can click the control 1121B, and in response to the operation, the mobile terminal 100-6 will stop running the game APP and display the desktop. And the mobile terminal will keep the game application locked for a preset time, such as within the day, that is, the user cannot continue to play the game on the day, and can only reopen the game at the specified time of the next day.

In other embodiments of the present application, when the mobile terminal 100-6 detects that the user has broken the rules, such as playing games for more than one hour at a time, the mobile terminal 100-6, in addition to restricting the game as shown in Figure 11L and providing the two options of exiting the game or unlocking the game; the mobile terminal 100-6 can also restrict any one or more functions including playing games, watching videos, reading novels, etc., or the mobile terminal 100-6 can also directly lock the screen of the mobile terminal 100-6, that is, not only the user is restricted from playing games, but also the user is restricted from using other functions (except emergency calls). If the user wants to continue using the mobile terminal 100-6, he must complete the unlocking task. If the user fails to complete the unlocking task, the mobile terminal 100-6 will automatically unlock after the specified time. The specified time is, for example, 24 hours later, or 0:00 am the next day. The embodiments of the present application do not limit this.

Next, FIG. 11M to FIG. 11R exemplarily show a series of user interface schematic diagrams of Xiao Ai using mobile terminal 100 - 6 to complete unlocking task 1 .

As shown in FIG11M , the user interface 112 displays a window 1122. Window 1122 displays a method for prompting the user to select an unlocking game, and options corresponding to unlocking task 1, such as option 1122A and option 1122B. Option 1122A and option 1122B are specifically: mobile terminal 100-6 displays option 1122A corresponding to tasks such as studying for 1 hour and running for 1 hour, and displays option 1122B corresponding to, for example, Xiaozhong's fingerprint and Xiaohua's fingerprint, according to unlocking task 1 of mobile terminal 100-6 set by Xiaozhong in mobile terminal 100-1 as shown in FIG10H above.

At this time, Xiao Ai can unlock the game by clicking on any of the above options and using the corresponding unlocking method. For example, the mobile terminal 100-6 can detect an operation acting on option 1122A shown in FIG. 11M , and in response to the operation, the mobile terminal 100-6 will provide the user with a learning task or a sports task to detect whether the user has completed the task; or the mobile terminal will detect whether the user has completed the above task for unlocking the game in other sports equipment or learning equipment at home.

In the embodiment of the present application, the prompt information for executing the unlocking task exemplarily shown in FIGS. 11L to 11M may also be referred to as fourth prompt information.

In one possible implementation, assuming that mobile terminal 100-6 provides a learning task for the user, the mobile terminal detects an operation acting on option 1122A shown in FIG. 11M, and in response to the operation, displays a user interface provided by the learning APP as shown in FIG. 11N.

FIG. 11N exemplarily shows a schematic diagram of a user interface provided by the Learning A application.

As shown in Figure 11N, the user interface 116 displays options for learning courses, such as vocabulary topics, grammar topics, long and difficult sentence topics, and composition topics, etc., wherein each option corresponds to the learning progress of the course. For example, courses that have been studied will display a label such as completed, and courses to be studied will display a label such as not started.

The mobile terminal 100-6 can automatically open an unstarted course for the user to learn, or can also open a corresponding course for the user to learn according to the user's choice. For example, the mobile terminal 100-6 can detect the user's operation on the long and difficult sentence topic option, and in response to the operation, the mobile terminal 100-6 displays the above user interface as shown in FIG11O.

FIG. 110 exemplarily shows the details page corresponding to the long and difficult sentence special course provided by the learning A application.

As shown in FIG11O , the user interface 117 displays a portion of the course corresponding to the long and difficult sentence, which may include, for example, Example 1, Translation 1, Analysis 1, etc. In some embodiments of the present application, the mobile terminal 100-6 may determine that the user has started learning based on the time when the corresponding course details page is detected to be displayed. For example, the mobile terminal 100-6 may detect that the user has started learning when the user interface 117 shown in FIG11O is displayed, and record the time when FIG11O is displayed as the user's learning time.

Assume that the user Xiao Ai studies from the time 9:31 shown in FIG. 11O to the time 10:31 shown in FIG. 11P , then the mobile terminal 100 - 6 determines that the user's study time has reached 1 hour, and therefore, the mobile terminal 100 - 6 may display the prompt information as shown in FIG. 11P .

As shown in FIG11P , the user interface 117 displays another part of the course corresponding to the long and difficult sentences, for example, it may include: Example 9, Translation 9, Analysis 9, etc. The user interface 117 also displays a window 1171, in which a prompt message for prompting the user, such as "Congratulations, the learning time has reached 1 hour, and the learning task has been completed", as well as option 1171A and option 1171B are displayed. Option 1171A is used to close the current window 1171 and continue to display the user interface provided by Learning A shown in FIG11P; option 1171B is used to end the learning task and jump to the exercise task.

It can be understood that during the process of mobile terminal 100-6 displaying Figures 11O to 11P above, that is, during the user learning process, mobile terminal 100-6 can periodically authenticate the current user to ensure that the current user is Xiao Ai himself, and not someone else learning on behalf of Xiao Ai.

In one possible implementation, assuming that mobile terminal 100-6 provides exercise tasks for the user, mobile terminal 100-6 detects that the user acts on control 1171B in the user interface shown in FIG11P, and in response to the operation, displays the user interface provided by the AI fitness APP shown in FIG11Q.

FIG. 11Q exemplarily shows a user interface diagram provided by the AI fitness APP.

As shown in FIG11Q, the user interface 118 displays sports content, for example, the left area of the user interface 118 displays the image of the coach, and the right area displays the image of the user Xiao Ai. Specifically, the coach Owen displayed in the left area can demonstrate the action. In normal mode, Xiao Ai can learn sports according to the coach's picture guidance. The camera of the mobile terminal 100-6 records Xiao Ai's fitness movements in real time, that is, collects images of Xiao Ai exercising with the coach, and displays Xiao Ai's fitness situation in real time on the right side of the user interface. Xiao Ai can check whether her movements are standard based on the images fed back from the right area. At the same time, the mobile terminal 100-6 can also determine whether Xiao Ai's learning movements are the same as those of the coach, to determine whether Xiao Ai's movements are standard, and timely remind and guide Xiao Ai on how to standardize the movements, etc., so that users can exercise scientifically at home.

In other embodiments of the present application, when the mobile terminal 100-6 detects an operation on the control 1171B shown in FIG. 11P, in response to the operation, the mobile terminal 100-6 may first display the homepage of the AI fitness APP, and the user may enter the corresponding sports course and learn from the coach by selecting options such as recommendations, fitness area, shoulder and neck relaxation, yoga area, square dance area, etc. provided on the homepage; or the mobile terminal 100-6 may automatically block the learned sports and directly jump to the most recent unlearned course shown in FIG. 11Q above. The embodiments of the present application do not limit this.

Assume that the user Xiao Ai exercises from the time 10:31 as shown in FIG. 11Q to the time 11:31 as shown in FIG. 11R , then the mobile terminal 100 - 6 determines that the user's exercise time has reached 1 hour, and therefore, the mobile terminal 100 - 6 may display the prompt information as shown in FIG. 11R .

As shown in FIG11R , the user interface 118 displays an image of Xiao Ai exercising following the coach's instructions. A window 1181 is also displayed on the user interface 118. The window 1181 displays a prompt message for prompting the user, such as "Congratulations, this exercise time has reached 1 hour, the learning task has been completed! The game has been successfully unlocked", as well as options 1181A and 1181B. Option 1181A is used to close the current window 1181 and continue to display the user interface 118 provided by AI Fitness shown in FIG11R; option 1181B is used to end the exercise task and jump to the user interface provided by Game A.

It is worth noting that Figures 11N-11R above only illustrate how Xiao Ai uses mobile terminal 100-6 to complete unlocking task 1. In the embodiment of the present application, in addition to using mobile terminal 100-6 to complete unlocking task 1, Xiao Ai can also use other home devices to complete unlocking task 1. Other home devices can send the relevant data of Xiao Ai completing the shackles task to mobile terminal 100-6, and determine whether the user has completed unlocking task 1 based on the received data. The details are as follows:

Next, FIG. 11S to FIG. 11V exemplarily show a series of schematic diagrams of the mobile terminal 100 - 6 when Xiao Ai uses other household devices to complete the unlocking task 1 .

First, FIG. 11S to FIG. 11V exemplarily illustrate a series of schematic diagrams of the mobile terminal 100 - 6 when Xiao Ai uses a notebook to complete a learning task.

As shown in FIG. 11S , the user interface 112 is the same as that shown in FIG. 11M above. The description of FIG. 11M is also applicable to the description of FIG. 11S , which will not be repeated here.

At this time, Xiao Ai can unlock the game by clicking any option shown in Figure 11S and using the corresponding unlocking method. For example, mobile terminal 100-6 can detect an operation on option 1122A shown in Figure 11S, and in response to the operation, mobile terminal 100-6 will display a prompt message as shown in Figure 11T.

As shown in FIG11T , a window 1123 is displayed on the user interface 112, and the window 1123 displays prompt information for prompting the user how to complete unlocking task 1, as well as corresponding controls, such as controls 1123A and 1123B, wherein control 1123A is used to control the notebook to start, thereby providing learning services for Xiao Ai; and control 1123B is used to start the treadmill, thereby providing exercise services for Xiao Ai.

It can be understood that Figure 11T only exemplifies the prompt information of two types of unlocking tasks 1. In other embodiments of the present application, when Xiao Ai's home devices also include devices such as smart screens and bracelets, the mobile terminal 100-6 can also display prompt information for exercising using the smart screen and bracelet, etc. Xiao Ai can open applications such as AI fitness on the smart screen and exercise with the coach; or Xiao Ai can wear a bracelet to exercise, etc. This is not displayed in the embodiments of the present application.

In some embodiments of the present application, Xiao Ai can open the corresponding home device by clicking the control shown in FIG. 11T to complete the corresponding unlocking task 1, or Xiao Ai can directly operate the corresponding home device according to the prompt given in window 1123 to complete the corresponding unlocking task 1. The details are as follows:

Refer to Figure 11U, which exemplarily shows an application scenario diagram of Xiao Ai using a notebook to study.

The notebook can send the relevant data of Xiao Ai's learning, such as the start time and the end time of learning, or the information of the continuous learning time, to the mobile terminal 100-6, so that the mobile terminal 100-6 can determine whether Xiao Ai has completed the unlocking task 1. Assuming that Xiao Ai learns from 9:31 to 10:31, after the mobile terminal 100-6 receives the message indicating that Xiao Ai's learning time reaches 1 hour, the mobile terminal 100-6 will display the user interface as shown in FIG11V.

As shown in FIG. 11V , the label displayed in the control 1123A displayed in the user interface 112 changes from “Go to Study” as shown in FIG. 11T to “Completed”, thereby prompting the user that the task of studying for 1 hour has been completed.

Next, Figures 11W to 11X exemplarily illustrate a series of schematic diagrams of the mobile terminal 100-6 when Xiao Ai uses a treadmill to complete a sports task.

In some embodiments of the present application, Xiao Ai can start the treadmill by clicking the control 1123B shown in FIG. 11V to complete the running task, or Xiao Ai can directly operate the treadmill according to the prompt given in the window 1123 to complete the corresponding unlocking task 1. The details are as follows:

Refer to Figure 11W, which exemplarily shows a schematic diagram of an application scenario in which Xiao Ai uses a treadmill to run.

The treadmill can send the relevant data of Xiao Ai's exercise, such as the start time and the end time of the exercise, or the information of the continuous exercise time, to the mobile terminal 100-6, so that the mobile terminal 100-6 can determine whether Xiao Ai has completed the unlocking task 1. Assuming that Xiao Ai exercises from 9:31 to 10:31, after the mobile terminal 100-6 receives the message indicating that Xiao Ai's exercise time reaches 1 hour, the mobile terminal 100-6 will display the user interface as shown in Figure 11X.

As shown in FIG11X , the label displayed in the control 1123B displayed in the user interface 112 changes from “Go Exercise” shown in FIG11V to “Completed”, thereby prompting the user that the task of exercising for 1 hour has been completed. In addition, a prompt message such as “Congratulations, you have completed unlocking task 1” is also displayed.

In the embodiment of the present application, the information sent by the notebook and treadmill to the mobile terminal 100-6 after Xiao Ai completes the unlocking task can also be referred to as the fourth information. The above-mentioned notebook, treadmill and other devices belong to the second device.

In some embodiments of the present application, during the process of Xiao Ai performing unlocking task 1, if Xiao Ai uses the local electronic device, i.e., the mobile terminal 100-6, to perform the task, the mobile terminal 100-6 periodically performs identity authentication during the process of Xiao Ai performing the task to ensure that the user performing the unlocking task is Xiao Ai himself. The identity authentication method includes any one or more of the following: password authentication, facial authentication, fingerprint authentication, voiceprint authentication, iris authentication, and operation habit authentication, etc., which are not limited in the embodiments of the present application. If Xiao Ai collaborates with other home devices 200 to perform the unlocking task, then during the process of performing the unlocking task, other home devices may also periodically collect user data for identity authentication, perform identity authentication on the local side, and send the authentication result to the mobile terminal 100-6; or send the collected data together with the relevant data of the execution task to the mobile terminal 100-6, so that it can determine whether the user performing the unlocking task is Xiao Ai himself. Specifically, when Xiao Ai uses a notebook to perform a learning task, the notebook can call a camera to collect the user's facial information, or call a fingerprint sensor to collect the user's fingerprint information, or call a microphone to collect the user's voice, etc., and verify the user's identity based on the collected information. Only after the identity authentication is passed and the unlocking task is completed, the mobile terminal 100 - 6 will be unlocked, or the locking function will be released.

Optionally, when the mobile terminal 100-6 detects that the user has completed unlocking task 1 and the identity authentication has passed, the mobile terminal 100-6 or other devices such as a notebook or a treadmill need to send the result of the user's execution of the unlocking task to the parent's mobile terminal, such as mobile terminal 100-1, and the mobile terminal 100-6 can be unlocked for Xiao Ai only after the parent's review and the review is passed. For example, the mobile terminal 100-6 can send the results of Xiao Ai's learning, such as the answers to the questions or the accuracy of the questions, and the results of Xiao Ai's learning from the fitness coach, such as the standard of movements and the completion rate, to the mobile terminal 100-1, and then Xiao Ai can decide whether to pass the unlocking request for Xiao Ai based on the quality of Xiao Ai's completion of the task. When the mobile terminal 100-6 receives the message sent by the mobile terminal 100-1 indicating that the review has passed, the mobile terminal 100-6 will unlock the screen or unlock some functions; otherwise, the mobile terminal 100-6 will keep the screen locked or some functions locked.

In an embodiment of the present application, the result of the user performing the unlocking task sent to the mobile terminal 100-1 by the mobile terminal 100-6 or other devices such as a notebook or a treadmill can also be referred to as the fourth information, and the message indicating that the review has passed and received by the mobile terminal 100-6 and sent by the mobile terminal 100-1 can also be referred to as the fifth message.

It can be understood that the method of completing the unlocking task in the mobile terminal 100-6 described in Figures 11M to 11R above can be combined with the method of completing the unlocking task in other home devices described in Figures 11S to 11X. That is, Xiao Ai can complete part of the tasks in unlocking task 1, such as the learning task, in the mobile terminal 100-6, and then complete another part of the tasks in unlocking task 1, such as the learning task, in a home device such as a treadmill. In this way, the mobile terminal 100-6 can determine that Xiao Ai has completed unlocking task 1 and then unlock the mobile terminal 100-6.

The above only introduces the UI implementation example in which Xiao Ai unlocks the mobile terminal 100-6 by completing unlocking task 1. In another possible implementation method, Xiao Ai may choose to unlock the mobile terminal 100-6 by completing unlocking task 2. The details are as follows:

Next, FIG. 11Y-FIG 11Z-5 exemplarily show a series of user interface schematic diagrams of Xiao Ai using mobile terminal 100 - 6 to complete unlocking task 2 .

As shown in FIG. 11Y , the user interface 112 is the same as that shown in FIG. 11M above. The description of FIG. 11M is also applicable to the description of FIG. 11Y , and will not be repeated here.

At this time, Xiao Ai can unlock the game by clicking any of the above options through the corresponding unlocking method. For example, the mobile terminal 100-6 can detect the operation of option 1122B shown in Figure 11Y, and in response to the operation, the mobile terminal 100-6 will turn on the fingerprint recognition function, as shown in Figure 11Z-1.

As shown in FIG11Z-1, a window 1124 is displayed in the user interface 112, and the window 1124 displays prompt information corresponding to the unlocking task 2 and the fingerprint collection area 1125. The prompt information is the unlocking task 2 of the mobile terminal 100-6 set by the mobile terminal 100-6 in the mobile terminal 100-1 according to Xiaozhong shown in FIG10H above, for example, "Please enter Xiaozhong's fingerprint and Xiaohua's fingerprint to unlock."

At this time, Xiao Ai can request his father Xiao Zhong and his mother Xiao Hua to input their fingerprint information to the mobile terminal 100 - 6 , as follows:

As shown in FIG. 11Z-2 , Daddy Xiaozhong inputs fingerprint information into the fingerprint collection area 1125 , and after the mobile terminal 100 - 6 detects Daddy Xiaozhong's fingerprint, it displays a user interface as shown in FIG. 11Z-3 .

As shown in FIG. 11Z-3 , a window 1124 is displayed in the user interface 112 , and the prompt information displayed in the window 1124 changes from the prompt information shown in FIG. 11Z-1 above to a prompt information such as “Xiaozhong’s fingerprint has been successfully identified, please continue to enter Xiaohua’s fingerprint to unlock”.

As shown in FIG. 11Z-4 , mother Xiaohua inputs fingerprint information into the fingerprint collection area 1125 . After the mobile terminal 100 - 6 detects the fingerprint of mother Xiaohua , it displays the user interface shown in FIG. 11Z-5 .

As shown in FIG. 11Z-5 , the user interface 112 displays a window 1124 , and the prompt information displayed in the window 1124 changes from the prompt information shown in FIG. 11Z-3 above to a prompt information such as “Xiaozhong’s fingerprint and Xiaohua’s fingerprint have been identified, and unlocking is successful.”

It is understandable that the present embodiment of the application does not display the order in which Dad Xiaozhong and Mom Xiaohua input fingerprint information to the mobile terminal 100-6. That is, when Xiaoai selects unlocking task 2 to unlock the mobile terminal 100-6, he can first find Dad Xiaozhong to input fingerprint information, and then find Mom Xiaohua to input fingerprint information as described above, or he can first find Mom Xiaohua to input fingerprint information, and then find Dad Xiaozhong to input fingerprint information. The present embodiment of the application does not limit this.

In other embodiments of the present application, Xiao Ai can also remotely request Xiao Zhong to unlock it, that is, the mobile terminal 100-1 can also receive the fifth operation input by Xiao Zhong and send the authorization instruction, i.e., the fifth information, to the mobile terminal 100-6 to indicate that Xiao Ai can continue to use some or all of the functions of the mobile terminal 100-6.

After the operation of setting the usage rules of mobile terminal 100-6 in mobile terminal 100-1 as described in Figures 10A to 10H above, mobile terminal 100-1 will collect user data according to the operation input by the user, that is, the application program collects data as described in the data bank system above, specifically, the smart home application program collects the data of the usage rules of mobile terminal 100-6. Then mobile terminal 100-1 opens the usage rules to mobile terminal 100-6 across devices.

In the process of the user using the mobile terminal 100-6 introduced in Figures 11A to 11K above, the mobile terminal 100-6 presents prompt information of the usage rules and prompt information after breaking the usage rules. This reflects the data value presentation introduced in the data bank system in the previous article. Specifically, the smart home application collects data on the usage rules of the user using the mobile terminal 100-6 and combines the formulated usage rules to output corresponding prompt information.

The process of the user breaking the rules of using the mobile terminal and executing the unlocking task and completing the unlocking as described in Figures 11L-11Z-5 above involves the collection, processing, opening, and value presentation of the user data described in the data bank system above. Specifically, the home device collects the data of the user executing the unlocking task, processes the relevant data of the unlocking task into the indication information of whether the unlocking task is completed, and opens the processed data to the mobile terminal, and presents the prompt information of whether the user has completed the unlocking task on the mobile terminal.

The methods of collecting, processing, opening up, and presenting the value of user data will not be elaborated here. For details, please refer to the introduction to the data bank system in the previous article.

In family life, the management, control and use of home devices cannot be separated from user operations. Therefore, in order to take care of the family, users need to spend a lot of energy to manage, control and use home devices. Especially when the user is away from home, home devices are unattended, which can easily cause damage to the equipment, waste of water and electricity resources, the family hygiene environment is persecuted by bad weather, and family safety is threatened by others, etc.

In order to reduce the pressure of users in taking care of the household and save users' time, the present application provides a method for controlling household devices, whereby a mobile terminal obtains the user's status information and information about events recorded by multiple applications in the mobile terminal, and controls the household devices to perform operations corresponding to the above events according to the user's status and/or the information about the events recorded in the applications, thereby realizing intelligent management, control, and use of household devices without user operation, thereby creating a healthy, safe, comfortable, low-carbon, and convenient personalized home life.

The above user statuses include: leaving home, returning home, sleeping, studying at home, working at home, exercising at home, getting off the car, etc.

The above-mentioned multiple applications can be: event memo applications such as calendars, alarm clocks, notes, memos, etc.; or life service applications such as applications for making appointments or purchasing medical treatment, dining, travel, and hairdressing services; or applications that provide public accounts and mini-program services, which can provide users with appointments or purchase medical treatment, dining, travel, hairdressing services, etc.; or weather forecast applications, which can provide users with weather forecast services; or online conference applications, which can provide users with meeting scheduling services; or online course learning applications, which can only allow users to book or purchase a course and then conduct online learning. The above-mentioned applications can be system applications or third-party applications. The embodiments of the present application are not limited to this.

The method for the mobile terminal to obtain the user status includes the following two methods:

1. The mobile terminal actively detects the user status. The details are as follows:

Method 1: Detect that the mobile terminal automatically disconnects the communication connection with the home device, and determine that the user status is away from home. The communication connection with the home device includes: a Wi-Fi connection with a home wireless router, or a Bluetooth connection with, for example, a home audio system, a sweeping robot, etc.; the embodiment of the present application does not limit the technology used for the communication connection, nor does it limit the type of the connection object, i.e., the home device. For details, please refer to the detailed description of the UI embodiment below, which will not be repeated here.

It is worth noting that the above automatic disconnection means that the disconnection operation is not triggered by user input, but the communication distance between the mobile terminal and the home device is detected to exceed the range, and the communication connection is automatically disconnected. However, since some users have good device usage habits, in order to reduce power consumption, they will actively disconnect the communication connection with the home device before leaving home. Therefore, in other embodiments of the present application, after the mobile terminal disconnects the communication connection with the home device based on the detected user operation, it can also be determined that the user is away from home.

Method 2: The mobile terminal determines the user status based on the information of the events recorded in the local application. Specifically, the user is judged to be in a sleeping state based on the sleeping time and waking time recorded in the alarm clock application; or the user is judged to be away from home based on the time information and location information of the user's dining reservation recorded in the life service application; or the user is judged to be in a home office state based on the meeting time recorded in the online meeting application. For details, please refer to the detailed description of the UI implementation example below, which will not be repeated here.

Method 3: The mobile terminal determines the user's status based on the messages received from other home devices. For example, the mobile terminal receives a message from a smart door lock that the user has left home, and determines that the user is away from home; or the mobile terminal receives data from a home monitoring device and determines that the user is away from home, home, sleeping, studying, exercising, etc.; or the mobile terminal receives information from a car computer that the vehicle is locked and determines that the user has gotten off the vehicle. For details, please refer to the detailed description of the UI implementation example below, which will not be repeated here.

It can be understood that the above exemplifies three specific implementation methods of the mobile terminal actively detecting the user status. In the embodiment of the present application, the mobile terminal can combine any two or three of the above three methods to determine the user status.

When the mobile terminal needs to combine the detection results of method 1 and method 2 to determine the user status. Taking the away state as an example, when the mobile terminal detects that the mobile terminal is disconnected from the home device according to method 1, it cannot directly determine that the user's status is away from home. The mobile terminal also needs to detect that the event information recorded in the application contains an event indicating that the user will leave home, and the difference between the time when the user leaves home and the time when the mobile terminal disconnects the communication connection is less than the threshold. At this time, the mobile terminal can determine that the user's status is away from home.

When the mobile terminal needs to combine the detection results of method 1 and method 3 to determine the user status. Still taking the away state as an example, when the mobile terminal detects that the mobile terminal is disconnected from the home device according to method 1, it cannot directly determine that the user's status is away from home. The mobile terminal also needs to detect that the information fed back by other home devices contains information indicating that the user is away from home, and the difference between the time when the user is away from home and the time when the mobile terminal is disconnected from the communication connection is less than the threshold. At this time, the mobile terminal can determine that the user's status is away from home.

When the mobile terminal needs to combine the detection results of method 2 and method 3 to determine the user status. Taking the sleep state as an example, when the mobile terminal detects that the event information recorded in the application contains an event indicating that the user is sleeping according to method 2, it cannot directly determine that the user is sleeping in the corresponding event. The mobile terminal also needs to detect that the information feedback from other home devices contains information indicating that the user is sleeping, and the difference between the sleep time in the information and the sleep time recorded in the application is less than the threshold. At this time, the mobile terminal can determine that the user is away from home.

When the mobile terminal needs to combine the detection results of method 1, method 2 and method 3 to determine the user status. Taking the away-from-home status as an example, when the mobile terminal detects that the mobile terminal is disconnected from the home device according to method 1, it cannot directly determine that the user's status is away from home. The mobile terminal also needs to detect that the event information recorded in the application contains events indicating that the user will leave home, and the time difference between the time the user leaves home and the time when the mobile terminal disconnects the communication connection is less than a threshold. In addition, the mobile terminal also needs to detect that the information fed back by other home devices contains information indicating that the user is away from home, and the time difference between the time the user leaves home and the time when the mobile terminal disconnects the communication connection is less than a threshold. At this time, the mobile terminal can determine that the user's status is away from home.

2. The user inputs the user status to the mobile terminal. The details are as follows:

The mobile terminal determines the user's status according to the received user input operation indicating the user's status.

For example, the user selects operations such as leaving home, returning home, studying, exercising, etc. in the smart home APP. The mobile terminal can determine the user status based on these operations. For details, please refer to the detailed description of the UI implementation example below, which will not be repeated here.

The mobile terminal may obtain information about each event in multiple applications in the following two ways:

1. After receiving the operation of the user inputting information in multiple applications, the mobile terminal obtains the information of the events in the multiple applications. The operation may be an operation of the user adding an event by inputting event information in event memo applications such as calendars and memos, or an operation of the user booking a service described above in life service applications or online course learning applications. For details, please refer to the detailed description of the method embodiment below, which will not be repeated here.

2. The mobile terminal directly obtains information about events in the application without receiving user operations. For example, the mobile terminal can directly obtain information about events provided by weather forecast applications, such as information about a certain time and place or the occurrence of bad weather.

The above events include: travel, medical treatment, dining, hairdressing, home study, home office, home fitness, and severe weather, etc.

The information of the above events includes: the name, time and place of the event.

Next, still taking a small or medium-sized family as an example, the control method of the above-mentioned home device is introduced in combination with the UI implementation example.

First, referring to FIG. 12A to FIG. 12V , FIG. 12A to FIG. 12V exemplarily illustrate a series of user interface schematic diagrams in which a mobile terminal actively detects a user status.

12A-12B exemplarily show a set of user interface schematic diagrams when the mobile terminal 100 - 1 uses method 1 to determine the user status.

Referring to FIG. 12A , FIG. 12A exemplarily shows a desktop displayed by the mobile terminal 100 - 1 .

As shown in FIG. 12A , the user interface 121 is both a desktop displayed by the mobile terminal 100 - 1 and a main interface for displaying an application menu.

The user interface 121 may include: a status bar 1211, a calendar indicator and a weather indicator, a page indicator, a tray with common application icons, and other application icons. Among them:

The status bar 1211 may include: one or more signal strength indicators 1211A of mobile communication signals (also referred to as cellular signals), one or more signal strength indicators 1211B of wireless fidelity (Wi-Fi) signals, a battery status indicator 1211C, and a time indicator 1211D.

The calendar indicator and weather indicator can be used to indicate the current time, such as date, day of the week, hour and minute information, etc., and can be used to indicate the weather type, such as cloudy to sunny, light rain, etc., and can also be used to indicate information such as temperature.

The page indicator can be used to indicate which page the user is currently browsing. In the embodiment of the present application, the application icons can be distributed on multiple pages, and the user can slide left and right to browse the application icons on different pages.

A tray 314 with commonly used application icons may display: a phone icon 314A, a text message icon 314B, a browser icon 314C, and a camera icon 314D.

Other application icons may include: weather icon, Huawei store icon, smart home icon, sports and health icon, memo icon, calendar icon, life service application icons, and online meeting application icons.

In some embodiments, the user interface 121 shown in Figure 12A is both the desktop of the mobile terminal 100-1 and the home screen interface. It is understandable that Figure 12A merely illustrates a user interface of the mobile terminal 100-1 and should not constitute a limitation on the embodiments of the present application.

It is worth noting that the specific time of the time indicator 1211D displayed in the user interface 121 shown in FIG12A is 8:00, and the Wi-Fi signal strength indicator 1211B represents that the mobile terminal 100-1 is connected to a wireless router or a hotspot opened by other devices at this time. Assuming that the mobile terminal 100-1 establishes a Wi-Fi connection with the router at home at this time, the mobile terminal 100-1 can determine that Xiaozhong is at home at this time.

In some embodiments of the present application, when the communication distance between the mobile terminal 100-1 and the home router exceeds a threshold, the mobile terminal 100-1 will disconnect the Wi-Fi connection from the home wireless router. At this time, the mobile terminal 100-1 can determine that the owner's status has switched to away from home.

In other embodiments of the present application, when the mobile terminal 100-1 detects that the Wi-Fi connection with the home wireless router is disconnected, the mobile terminal 100-1 will not directly determine that the user is away from home, and the mobile terminal 100-1 may display a prompt message as shown in Figure 12B.

12B , which exemplarily shows a user interface diagram when the mobile terminal 100 - 1 detects that the mobile terminal 100 - 1 is disconnected from the home wireless router.

As shown in FIG12B , the time indicator 1211D in the status bar 1211 displayed in the user interface 121 specifically displays the time as 8:30, and no longer includes the Wi-Fi signal strength indicator 1211B shown in FIG12A above, but is replaced by Internet access icons such as 5G, 4G, 3G, and 2G. In addition, a prompt window 1212 is also displayed in the user interface 121. The prompt window 1212 displays a prompt message such as "The WiFi connection has been disconnected, may I ask if you have left home", as well as controls 1212A and 1212B. Control 1212A is used to confirm that Xiaozhong has left home, and control 1212B is used to deny that the user has left home.

At this time, if Xiaozhong is sure that he has left home, he can click on the control 1212A in Figure 12B. The mobile terminal 100-1 detects the operation and determines, in response to the operation, that Xiaozhong's current state is away from home.

It is understandable that when the mobile terminal 100-1 does not detect an operation on the control 1212B and does not detect an operation on the control 1212A within a preset time, the mobile terminal 100-1 may assume that the user has left home.

It can be understood that Figure 12B only exemplifies one form of prompt information. The above-mentioned window 1212 can be displayed in the desktop shown in Figure 12B, and can also be displayed in other user interfaces such as the interface provided by the smart home APP. The embodiment of the present application does not limit this.

It is understandable that, in addition to being displayed in the form of window 1212, the above prompt information can also be displayed in a pull-down notification bar or in the negative one screen, and the embodiment of the present application does not limit this.

It is understandable that the above FIG. 12A-FIG. 12B merely exemplarily shows that the mobile terminal 100-1 is disconnected from the home wireless router to determine whether the user is away from home. In addition to the above method, the mobile terminal 100-1 can also determine whether the user is away from home by detecting whether the Bluetooth connection with its home devices, such as home smart speakers, is disconnected. The details are as follows:

12C-12D exemplarily show another set of user interface schematic diagrams when the mobile terminal 100 - 1 uses method 1 to determine the user status.

As shown in FIG12C , the user interface 121 is both the desktop displayed by the mobile terminal 100-1 and the main interface for displaying the application menu. The user interface 121 shown in FIG12C is similar to the user interface 121 shown in FIG12A above, so the description of FIG12A also applies to the description of FIG12C , but FIG12C and FIG12A above also have the following differences: the status bar 1211 includes one or more signal strength indicators 1211A of the mobile communication signal (also referred to as the cellular signal), a Bluetooth indicator 1211E, a battery status indicator 1211C, and a time indicator 1211D.

It is worth noting that the specific time of the time indicator 1211D displayed in the user interface 121 shown in FIG12C is 8:00, and the Bluetooth indicator 1211F represents that the mobile terminal 100-1 turns on the Bluetooth function or establishes a Bluetooth connection with other devices at this time. Assuming that the mobile terminal 100-1 establishes a Bluetooth connection with a home device such as a home smart speaker at this time, the mobile terminal 100-1 can determine that Xiaozhong is at home at this time.

In some embodiments of the present application, when the communication distance between the mobile terminal 100-1 and the home smart speaker exceeds a threshold, the mobile terminal 100-1 will disconnect the Bluetooth connection with the home smart speaker. At this time, the mobile terminal 100-1 can directly determine that the status of the owner Xiaozhong has switched to leaving home.

In other embodiments of the present application, when the mobile terminal 100-1 detects that a Bluetooth connection with a home smart speaker is established or disconnected, the mobile terminal 100-1 will not directly determine that the user is away from home, and the mobile terminal 100-1 may display a prompt message as shown in FIG. 12D.

Referring to FIG. 12D , FIG12D exemplarily shows a user interface diagram when the mobile terminal 100 - 1 detects that the Bluetooth connection with the home smart speaker is disconnected.

As shown in FIG12D , the time indicator 1211D in the status bar 1211 displayed in the user interface 121 specifically displays the time as 8:30, and no longer includes the Bluetooth indicator 1211F shown in FIG12C above. In addition, a prompt window 1213 is also displayed in the user interface 121. The prompt window 1213 displays a prompt message such as "The Bluetooth connection with the smart speaker has been disconnected. May I ask if you have left home?", as well as controls 1213A and 1213B. Control 1213A is used to confirm that Xiaozhong has left home, and control 1213B is used to deny that the user has left home.

At this time, if Xiaozhong is sure that he has left home, he can click on the control 1213A in Figure 12D. The mobile terminal 100-1 detects the operation and determines, in response to the operation, that Xiaozhong's current state is away from home.

It is understandable that when the mobile terminal 100-1 does not detect an operation on the control 1213B and does not detect an operation on the control 1213A within a preset time, the mobile terminal 100-1 may assume that the user has left home.

It can be understood that Figure 12D only exemplifies one form of prompt information. The above-mentioned window 1213 can be displayed in the desktop shown in Figure 12D, and can also be displayed in other user interfaces such as the interface provided by the smart home APP. The embodiment of the present application does not limit this.

It is understandable that, in addition to being displayed in the form of window 1213, the above prompt information can also be displayed in a pull-down notification bar or in the negative one screen, and the embodiment of the present application does not limit this.

12E-12I exemplarily show a set of user interface schematic diagrams when the mobile terminal 100 - 1 uses method 2 to determine the user status.

12E-12H are schematic diagrams of the operation interface for a user to book travel services in a life service application.

As shown in FIG12E, when the mobile terminal 100-1 detects the user's operation on the icon of the life service application in the user interface 121 shown in FIG12E, in response to the operation, the mobile terminal 100-1 displays the user interface 122 provided by the life service application shown in FIG12F. The user interface 121 shown in FIG12E is the same as the user interface 121 shown in FIG12A above, and the text description of the user interface 121 in FIG12A is also applicable to the user interface 121 shown in FIG12E.

FIG. 12F exemplarily shows a schematic diagram of a user interface provided by a life service application in the mobile terminal 100 - 1 .

As shown in FIG. 12F , the user interface 122 may include a search bar 1221 , tabs 1222 , and a page navigation 1223 . The page navigation bar 1223 may include controls corresponding to sub-pages such as the home page, order, and mine.

Among them, the tab 1222 specifically includes options such as: hotel, air tickets, travel, taxi, food, movies, medical care, beauty and travel, etc. The user can reserve or purchase the corresponding service by clicking the above options. For example, the mobile terminal 100-1 can detect the user's operation of clicking the taxi option in the user interface 122. In response to the operation, the mobile terminal 100-1 can display the details page corresponding to the taxi service as shown in FIG12G.

As shown in FIG. 12G , the user interface 123 displays: a map 1231 including the user's current location, options 1232 , an input box 1233 , a frequently used address option 1234 , and controls 1235 .

Option 1232 is a departure location that users often use when taking a taxi. Users can re-edit the departure location by clicking on this option. Input box 1233 is used to enter a destination. Common address option 1234 is, for example, Xiao Ai's school. Control 1235 is used to add more options corresponding to common addresses.

When Xiaozhong wants to take a taxi from home to Xiaoai's school, Xiaozhong can click on the option corresponding to the common address corresponding to Xiaoai's school as shown in Figure 12G. In response to this operation, mobile terminal 100-1 displays the user interface shown in Figure 12H.

As shown in FIG. 12H , the user interface 123 displays: a map 1236 including a route from the departure point to the destination, an estimated price 1237 , and a control 1238 for calling a car immediately.

When the mobile terminal 100-1 detects an operation on the control 1238 shown in FIG12H, in response to the operation, the life service application will record the taxi-hailing event and send the event to the smart home APP, which obtains the time when the user leaves home based on the event, that is, determines when the user is away from home.

In other embodiments of the present application, after the mobile terminal 100-1 detects that the user inputs the operation for hailing a taxi as shown in Figures 12E-12H above, the mobile terminal 100-1 will not directly determine that the user's status is away from home. The mobile terminal 100-1 may first display the prompt information shown in Figure 12I.

Referring to FIG. 12I , FIG12I exemplarily shows a user interface diagram of prompt information displayed by the mobile terminal 100 - 1 according to an event of a user purchasing a travel service.

As shown in FIG12I , a prompt window 1239 is displayed in the user interface 123. The prompt window 1239 displays a prompt message such as “It is detected that you are taking a taxi from home to Xiao Ai’s school. Do you want to leave home?”, as well as a control 1239A and a control 1239B. The control 1239A is used to confirm that Xiao Ai has left home, and the control 1239B is used to deny that the user has left home.

At this time, if Xiaozhong is determined to leave home, he can click on the control 1239A in Figure 12I. The mobile terminal 100-1 detects the operation and, in response to the operation, determines that Xiaozhong's current state is away from home.

It is understandable that when mobile terminal 100-1 does not detect an operation on control 1239B and does not detect an operation on control 1239A within a preset time, mobile terminal 100-1 may assume that the user has left home.

It can be understood that Figure 12I only exemplifies one form of prompt information. The above-mentioned window 1239 can be displayed in the desktop shown in Figure 12I, and can also be displayed in other user interfaces such as the interface provided by the smart home APP. The embodiment of the present application does not limit this.

In other embodiments of the present application, when the mobile terminal 100-1 records event information that the user will carry Xiao Ai's schoolbag to Xiao Ai's school between 8:30 and 9:00, and the mobile terminal 100-1 detects that the user leaves home between 8:30 and 9:00, the mobile terminal 100-1 will also output prompt information to remind the user to carry Xiao Ai's schoolbag.

The above-mentioned 8:30-9:00 can also be called the fourth time period, the above-mentioned Xiao Ai's schoolbag can also be called the first item, the above-mentioned prompt information for carrying the schoolbag can also be called the sixth prompt information, and the information recorded in the above-mentioned mobile terminal 100-1 that the user will carry Xiao Ai's schoolbag to Xiao Ai's school between 8:30 and 9:00 can also be called the sixth information.

In other embodiments of the present application, when the mobile terminal 100-1 records event information about extreme weather such as rainfall and typhoons that will occur between 13:20 and 14:00, and the mobile terminal 100-1 detects that the user leaves home between 13:20 and 14:00, the mobile terminal 100-1 will also output prompt information to remind the user to bring protective equipment such as an umbrella or raincoat.

The above-mentioned 13:20-14:00 can also be called the fifth time period, and the above-mentioned reminder information for carrying protective equipment such as umbrellas and raincoats can also be called the seventh reminder information.

12J-12N exemplarily show another set of user interface schematic diagrams when the mobile terminal 100 - 1 uses method 2 to determine the user status.

12J-12N are schematic diagrams of the operation interface for a user to add a to-do event in a memo application.

As shown in FIG12J, when the mobile terminal 100-1 detects an operation performed by a user on an icon of a memo application such as a calendar in the user interface 121 shown in FIG12J, in response to the operation, the mobile terminal 100-1 displays a user interface 124 provided by the calendar shown in FIG12K. The user interface 121 shown in FIG12J is the same as the user interface 121 shown in FIG12A above, and the text description of the user interface 121 in FIG12A is also applicable to the user interface 121 shown in FIG12J.

FIG. 12K exemplarily shows a schematic diagram of a user interface provided by a calendar in mobile terminal 100 - 1 .

As shown in FIG12K , the user interface 124 displays a calendar 1241, a window 1242, a new schedule icon 1243, and subscription items recommended by the mobile terminal 100-1, such as the lunar calendar, etc. When the mobile terminal 100-1 detects in the user interface 124 that the user selects a certain day in the calendar 1241, such as September 26, and acts on the new schedule icon 1243, in response to the operation, the mobile terminal 100-1 displays the user interface 125 shown in FIG12L .

FIG. 12L exemplarily shows a user interface 125 displayed by the mobile terminal 100 - 1 .

As shown in FIG. 12L , the user interface 125 is a page to be edited for creating a new schedule. The user interface 125 may include a cancel icon 1251, a confirmation icon 1252, a page title such as creating a new schedule, a title input box, a location input box, a time input box, etc. Among them, the cancel icon 1251 can be used to close the page to be edited, and the confirmation icon 1252 can be used to confirm the edited to-do event. The title input box can be used to receive the title of the to-do event input by the user, the location input box can be used to receive the location of the to-do event input by the user, and the time input box can be used to receive the start and end time of the to-do event input by the user.

As shown in Figure 12M, mobile terminal 100-1 can detect that a user edits a to-do event in user interface 125. For example, the user can enter the name of the to-do event as an online meeting, the meeting location as Xiaozhong's home, and the meeting time as September 26, 14:30-15:30. After the editing is completed, mobile terminal 100-1 can also detect the user's operation on confirmation icon 1252. In response to this operation, the calendar of mobile terminal 100-1 receives the to-do event edited by the user, that is, the event added by the user to the calendar.

In an embodiment of the present application, in response to the mobile terminal 100-1 detecting an operation acting on the control 1252 shown in Figure 12M, the mobile terminal will not only remind the user to complete the meeting event at the scheduled time, but also send the meeting event to the smart home APP. The smart home APP obtains the user's home office time based on the meeting event, that is, determines when the user is in the home office state.

In other embodiments of the present application, after the mobile terminal 100-1 detects that the user inputs a series of operations shown in Figures 12J to 12M above, the mobile terminal 100-1 will not directly determine that the user's state is home. The mobile terminal 100-1 may first display the prompt information shown in Figure 12N.

Referring to FIG. 12N , FIG. 12N exemplarily shows a user interface diagram of prompt information displayed by mobile terminal 100 - 1 according to pending conference events.

As shown in FIG12N , a prompt window 1214 is displayed in the user interface 121. The prompt window 1214 displays a prompt message such as “It is detected that you have an online meeting at 14:30. Do you want to turn on the home office mode?”, as well as a control 1214A and a control 1214B. The control 1214A is used to confirm that the lieutenant general is in the home office state at 14:30, and the control 1214B is used to deny that the lieutenant general is in the home office state at 14:30.

It is worth noting that the time when the mobile terminal 100-1 displays the prompt information shown in Figure 12N can be immediately after detecting the addition of a to-do event, or it can be displayed at any time before the start event of the to-do event. For example, if the meeting start time is 14:30, the mobile terminal 100-1 can display the above prompt information ten minutes before the meeting start time, that is, 14:20. The embodiment of the present application does not limit this.

At this time, if Xiaozhong is determined to hold an online meeting at home, he can click on the control 1214A in Figure 12N. The mobile terminal 100-1 detects the operation and, in response to the operation, determines that Xiaozhong's status at 14:30 is working from home.

It is understandable that when mobile terminal 100-1 does not detect an operation on control 1214B and does not detect an operation on control 1214A within a preset time, mobile terminal 100-1 can default the status at 14:30 to the working from home status.

It can be understood that Figure 12N only exemplifies one form of prompt information. The above-mentioned window 1214 can be displayed in the desktop shown in Figure 12N, and can also be displayed in other user interfaces such as the interface provided by the smart home APP. The embodiment of the present application does not limit this.

12O to 12P exemplarily show a series of schematic diagrams in which the mobile terminal 100 - 1 detects that the user has determined that the user is away from home by remotely starting the vehicle.

FIG12O exemplarily shows a schematic diagram of a scenario in which Xiaozhong remotely controls the vehicle ignition through the mobile terminal 100 - 1 .

As shown in Fig. 12O, Xiao Zhong can use, for example, a remote vehicle control APP or a smart home APP in the mobile terminal 100-1 to remotely start the vehicle. The user interface diagram of remote vehicle start is not described in detail here.

In one possible implementation, when the mobile terminal 100 - 1 detects that the user inputs an operation for starting the vehicle, the mobile terminal 100 - 1 may directly determine that the user's status is away from home.

In another possible implementation, when the mobile terminal 100-1 detects that the user inputs an operation for starting the vehicle, the mobile terminal 100-1 does not directly determine that the user's status is away from home, but first displays a prompt message as shown in FIG. 12P.

FIG12P exemplarily shows a user interface diagram of the leaving home reminder information displayed when the mobile terminal 100 - 1 detects that the user inputs an operation to remotely start the vehicle.

As shown in FIG12P , the user interface 121 displays a prompt window 1215. The prompt window 1215 displays a prompt message such as “It is detected that you have remotely started the vehicle. Are you going to leave home?”, as well as a control 1215A and a control 1215B. The control 1215A is used to confirm that Xiaozhong has left home, and the control 1215B is used to deny that Xiaozhong has left home.

At this time, if Xiaozhong is sure that he has left home, he can click on the control 1215B in Figure 12D. The mobile terminal 100-1 detects the operation and determines, in response to the operation, that Xiaozhong's current state is away from home.

It is understandable that when the mobile terminal 100-1 does not detect an operation on the control 1215A and does not detect an operation on the control 1215B within a preset time, the mobile terminal 100-1 may assume that the user has left home.

It is understandable that when the mobile terminal 100-1 does not detect an operation on the control 1215B and does not detect an operation on the control 1215A within a preset time, the mobile terminal 100-1 may assume that the user has left home.

It can be understood that Figure 12P only exemplifies one form of prompt information. The above-mentioned window 1215 can be displayed in other user interfaces, such as the interface provided by the application, in addition to being displayed in the desktop shown in Figure 12P. The embodiment of the present application does not limit this.

It is understandable that, in addition to being displayed in the form of window 1215, the above prompt information can also be displayed in a pull-down notification bar or in the negative one screen, and the embodiment of the present application does not limit this.

12Q to 12R exemplarily show a series of schematic diagrams in which the mobile terminal 100 - 1 determines the user's away-from-home status based on the user's vehicle riding operation.

As shown in Fig. 12Q, Fig. 12Q exemplarily shows a schematic diagram of a barcode for taking a subway, bus, etc. displayed on the mobile terminal 100-1. When the barcode displayed on the mobile terminal 100-1 is scanned successfully, the mobile terminal 100-1 will display a user interface schematic diagram as shown in Fig. 12R.

As shown in FIG. 12R , the mobile terminal 100 - 1 displays a prompt message indicating that the ride was successful.

It is understandable that users can use mobile terminals to take buses or subways in more ways than just scanning the barcode provided by the ride APP as shown above. They can also use NFC card swiping to take the bus. This embodiment of the present application does not limit this.

In the embodiment of the present application, when the mobile terminal 100-1 detects the user's riding behavior, it can directly determine that the user is away from home.

In other embodiments of the present application, the information of events recorded in the application obtained by the mobile terminal 100-1 in mode 2 includes not only the information of events corresponding to the user purchasing a service or adding a to-do event in the application as shown in the above Figures 12E-12R, but also the information generated by the application itself, such as event information related to weather events generated by the weather forecast application itself. Therefore, the embodiments of the present application do not limit the information of events obtained by the mobile terminal from the application.

12S to 12V exemplarily illustrate another set of user interface schematic diagrams when the mobile terminal 100 - 1 uses method 3 to determine the user status.

When a home device such as a smart door lock detects that a user is leaving the house, the smart door lock can send information about the user leaving the house to the mobile terminal. After receiving the information, the mobile terminal will display a user interface diagram as shown in FIG12S.

FIG. 12S exemplarily shows a user interface diagram of the mobile terminal 100 - 1 receiving information sent by other home devices.

As shown in FIG12S , the user interface 121 displays a prompt message 1216. The prompt message 1216 displays a prompt message such as “Smart home receives a message”. It is understandable that the prompt message 1251 can be displayed in the user interface 121 shown in FIG2O , that is, the desktop, or in other user interfaces of the mobile terminal 100 - 1 , and the embodiment of the present application does not limit this.

In an embodiment of the present application, the user can jump to the message details page provided by the smart home APP by clicking the prompt information 1216 shown in Figure 12S.

It is understandable that after receiving a message sent by another home device, the mobile terminal 100-1 will not only display the prompt information shown in FIG. 12S, but also display the received message in the smart home APP. The details are as follows:

Refer to Figure 12T, which exemplarily shows a user interface diagram of the smart home APP when receiving a new message.

As shown in FIG. 12T , the user interface 81 is the same as the user interface 81 shown in FIG. 10B above. The above textual description of the user interface 81 in FIG. 10B is also applicable to the user interface 81 shown in FIG. 12T , and will not be repeated here.

The user interface 81 shown in FIG. 12T is different from the user interface shown in FIG. 10B above in that the icon 819 displayed in the user interface shown in FIG. 12T displays the number of unread messages, for example, "1". The user can click the icon 819 to open the received message details page and read the message received by the smart home APP.

When the mobile terminal 100-1 detects an operation on the prompt information 1216 in FIG. 12S, or detects an operation on the icon 819 shown in FIG. 12T, in response to the operation, the mobile terminal 100-1 will display the user interface diagram as shown in FIG. 12U.

As shown in FIG. 12U , the user interface 126 displays a message sent by the smart door lock, such as “the smart door lock detected that you left home at 16:00”.

In other embodiments of the present application, after receiving the prompt information shown in FIG. 12U , the mobile terminal 100-1 may directly determine that Xiaozhong is in the away state, or the mobile terminal 100-1 may not directly determine that Xiaozhong is in the away state. The mobile terminal 100-1 may first display the prompt information shown in FIG. 12V .

In the embodiment of the present application, the information sent by the smart door lock to the mobile terminal 100-1 can also be called the seventh information.

Referring to FIG. 12V , FIG. 12V exemplarily shows a user interface diagram of prompt information displayed by the mobile terminal 100 - 1 based on information sent by other home devices.

As shown in FIG12V , a prompt window 1217 is displayed in the user interface 121. The prompt window 1217 displays a prompt message such as “The smart door lock detects that you have just left the house. May I ask if you have left home?”, as well as controls 1217A and 1217B. Control 1217A is used to confirm that Xiaozhong is away from home at 16:00, and control 1217B is used to deny that Xiaozhong will be away from home at 16:00.

At this time, if Xiaozhong is determined to leave home, he can click on the control 1217A in Figure 12V. The mobile terminal 100-1 detects the operation and, in response to the operation, determines that Xiaozhong's status at 16:00 is away from home.

It is understandable that when mobile terminal 100-1 does not detect an operation on control 1217B and does not detect an operation on control 1217A within a preset time, mobile terminal 100-1 may default the status of the control at 16:00 to being away from home.

It can be understood that Figure 12V only exemplifies one form of prompt information. In addition to being displayed in the desktop shown in Figure 12V, the above-mentioned window 1217 can also be displayed in other user interfaces such as the interface provided by the smart home APP. The embodiment of the present application does not limit this.

In other embodiments of the present application, when the mobile terminal detects that the distance between itself and the home device exceeds a third value, such as 20 meters, it can also be confirmed that the user's status is away from home.

In the embodiment of the present application, when the smart door lock detects that the user opens the door, it can detect whether the user has brought a mobile phone. If the user does not have a mobile phone, the smart door lock can refuse to close the door or broadcast a voice prompt to remind the user to bring the mobile phone. The details are as follows:

When sensors in the smart door lock, such as a camera or a low-light sensor or an infrared device, detect that Xiaozhong walks into the door and is about to open it, the smart door lock can establish a communication connection with the mobile terminal 100-1, such as a Bluetooth connection or a WiFi connection, and use Bluetooth ranging or WiFi ranging to detect the distance between Xiaozhong's mobile terminal 100-1 and the smart door lock. Before the smart door lock detects that Xiaozhong closes the door, if the distance between the smart door lock and the mobile terminal 100-1 is less than a second value, such as 1 meter, the smart door lock outputs a fifth prompt message for prompting the user to carry the mobile terminal 100-1, or triggers the mobile terminal 100-1 and/or other home devices to output the fifth prompt message for prompting the user to carry the mobile terminal 100-1. When the smart door lock detects that the ninth operation opens the door lock, and the smart door lock detects that the distance between it and the mobile terminal 100-1 gradually increases, the smart door lock can determine that Xiao Zhong does not bring the mobile terminal 100-1, and the smart door lock can perform any one or more of the following operations: refuse to lock the door, or output the fifth prompt information for prompting the user to carry the mobile terminal 100-1, or trigger the mobile terminal 100-1 and other home devices to output the fifth prompt information for prompting the user to carry the mobile terminal 100-1.

In other embodiments of the present application, it is also possible to detect whether the user is carrying the mobile terminal 100 - 1 through home devices such as a camera.

In one possible implementation, when the smart door lock determines that Xiaozhong is not wearing a mobile phone when going out, the door may not be closed, thereby achieving the purpose of reminding Xiaozhong that he is not wearing the mobile terminal 100-1.

In another possible implementation, when the smart door lock determines that Xiaozhong goes out without carrying a mobile phone, it can output a prompt tone to remind Xiaozhong to carry the mobile terminal 100-1.

In other embodiments of the present application, when the smart door lock detects that a small person is approaching and the distance between the small person and the smart door lock is less than a second value, for example, 1 meter, the smart door lock can output the fifth prompt information in advance to remind the user to bring a mobile phone, ID, car keys, etc., or trigger the mobile terminal 100-1 and other home devices to output the fifth prompt information for reminding the user to bring the mobile terminal 100-1.

Similar to leaving home, when the vehicle receives the tenth operation input by the user, the vehicle door opens, and the vehicle door can detect the distance from the mobile terminal 100-1. When the vehicle door detects that the distance between it and the mobile terminal 100-1 is gradually increasing, the vehicle can determine that Xiaozhong did not bring the mobile terminal 100-1, and the vehicle can perform any one or more of the following operations: refuse to lock the door, or output the eighth prompt information for prompting the user to carry the mobile terminal 100-1, or trigger the mobile terminal 100-1 and or other home devices to output the eighth prompt information for prompting the user to carry the mobile terminal 100-1.

When the vehicle determines that the distance between the mobile terminal 100-1 and the door is less than the fourth value, the eighth prompt information can be output in advance to remind the user to take personal belongings such as the mobile terminal 100-1 and the elderly and children.

12W to 12Z exemplarily show a series of user interface schematic diagrams displayed by the mobile terminal 100 - 1 when the user actively inputs the user status to the mobile terminal.

As shown in FIG. 12W , the user interface 61 displayed by the mobile terminal 100 - 1 is similar to the user interface 61 shown in FIGS. 6K , 8F , and 8J above. The description of FIG. 6K in the above text is also applicable to the description of FIG. 12W , which will not be repeated here.

Mobile terminal 100 - 1 detects an operation of the user clicking on control 615C. In response to this operation, mobile terminal 100 - 1 displays user interface 126 as shown in FIG. 12X .

As shown in FIG12X, the user interface 126 is a smart scenario page, which specifically includes: controls 1261A and 1261B. Controls 1261A and 1261B are sub-page navigation controls of the smart scenario page, and the user can switch the "recommended" sub-page and the "my" sub-page in the smart scenario page respectively by operating controls 1261A and 1261B.

It is understandable that at this time, the user interface 126 displays the corresponding "recommendation" subpage of the control 1261A by default. This subpage displays options corresponding to one or more smart scenarios contained in the one-click execution module. For example, option 1262, option 1263, and option 1264, etc. The above options are respectively the options corresponding to smart scenarios such as leaving home with peace of mind, returning home warmly, and cleaning and purification provided by the smart home APP by default. The user can click any of the above options to make the mobile terminal 100-1 execute the smart scenario with one click, and specifically execute the operation of linking and controlling home devices.

In one possible implementation, the mobile terminal 100-1 detects the user's operation on option 1262 shown in FIG. 12X above, and in response to the operation, the mobile terminal 100-1 can determine that the user is away from home. In addition, the mobile terminal 100-1 will also apply the smart scene mode of leaving home with peace of mind, that is, linkage control of home devices to perform corresponding operations, for example, the mobile terminal 100-1 can control the TV, air conditioner, smoke stove, kitchen appliances, air purifier, fresh air machine, humidifier dehumidifier, sweeper, door lock, electric curtain, light, etc. in the home device to shut down.

In another possible implementation, the mobile terminal 100-1 may detect an operation for opening the away-from-home scenario in the My subpage, and determine that the user is in the away-from-home state.

As shown in FIG. 12Y , the user interface 126 displayed by the mobile terminal 100 - 1 is similar to the user interface 126 shown in FIG. 12X in the previous text. The description of FIG. 12X in the previous text is also applicable to the description of FIG. 12X and will not be repeated here.

Mobile terminal 100-1 can detect the user's operation on control 1261B shown in Figure 12Y. In response to the operation, mobile terminal 100-1 can switch the sub-page in the smart scenario page from the recommended sub-page to the "My" sub-page shown in Figure 12Z.

As shown in Figure 12Z, the user interface 126 displays a "My" sub-page. The "My" sub-page includes options corresponding to one or more smart scenarios contained in the user-defined one-key execution module, such as option 1265, option 1266, and option 1267. The above options are options corresponding to user-defined smart scenarios such as working from home, dreaming at dawn, and long-term business trips; the "My" sub-page also includes options corresponding to one or more smart scenarios corresponding to the user-defined smart orchestration module, such as option 1268 and option 1269. The above options are options corresponding to user-defined smart scenarios such as closing the door lock when leaving home and turning on the air conditioner when the temperature is high; the "My" sub-page also includes a control 1261C for adding a one-key execution or smart scenario under the smart orchestration module.

At this time, the mobile terminal 100-1 detects the user's operation on option 1267 shown in FIG. 12Z. In response to the operation, the mobile terminal 100-1 can determine that the user is away from home. In addition, the mobile terminal 100-1 will also apply the intelligent scenario mode for long-term business trips, that is, linkage control of home devices to perform corresponding operations, for example, the mobile terminal 100-1 can control the TV, refrigerator, air conditioner, smoke stove, kitchen appliances, air purifier, fresh air machine, humidifier dehumidifier, sweeper, door lock, electric curtain, electric light, etc. in the home device to shut down, turn on the monitoring equipment such as the camera in the smart door lock, and the operation of the camera in the house.

Based on the introduction of the user interface related to the method for a mobile terminal to detect user status in Figures 12A-12Z above, the following introduces a user interface schematic diagram of the mobile terminal automatically controlling home devices based on the user status and/or event information obtained in the application.

In the embodiment of the present application, the basis for the mobile terminal to control the home device to perform the corresponding operation includes the following three:

1. The mobile terminal can control home devices to execute corresponding events based only on the user status.

2. The mobile terminal executes the corresponding event according to the user status and the event information recorded in the application.

3. The mobile terminal controls the home devices to perform corresponding functions only according to the information of the events recorded in the application.

First, a method is introduced in which a mobile terminal controls a home device to execute a corresponding event only according to a user state, as follows:

The mobile terminal pre-stores the relationship between user status and smart scenarios. When the user forgets to actively turn on the corresponding smart scenario, the mobile terminal can apply the corresponding smart scenario mode according to the user's status to avoid the situation where home devices are poorly managed due to user forgetfulness.

The corresponding relationship between the user's status and the smart scenario can be, for example: when the user's status is away from home, the smart home app will apply the corresponding smart scenario, such as the mode of leaving home with peace of mind; when the user's status switches from leaving home to returning home, the smart home app will apply the mode of leaving home with peace of mind; when the user's status is working from home, the smart home app will apply the mode of working from home; when the user's status is away from home and is away from home for a long time (for example, away from home for more than a week), the smart home app will apply the long-term business trip mode; when the user's status is in sleep mode, the smart home app will apply the mode of dreaming until dawn. The embodiments of the present application do not limit the types of user status and smart scenarios, and will not be described one by one here.

As mentioned above, the smart scenario can be provided by the smart home app by default, such as "leave home with peace of mind", "return home warmly", "clean and purified", etc. as exemplified in Figure 12X above, or it can be a user-defined scenario mode, such as "work from home", "dream till dawn", "long-term business trip", etc. as exemplified in Figure 12Z above. The embodiments of the present application do not limit this.

13A-13C , which exemplarily illustrate schematic diagrams of user interfaces in which a mobile terminal controls home devices according to a user status.

FIG. 13A exemplarily shows a user interface diagram provided by a smart home app.

As shown in FIG13A , the user interface 61 is the home page provided by the smart home APP. At this time, the device display area 614 displays the status of each home device, for example, air conditioner A, smart light A, air conditioner B, etc. are in the on state.

When the mobile terminal 100 - 1 detects that the user status is away from home, the mobile terminal 100 - 1 will apply the smart scenario corresponding to leaving home according to the correspondence between the user status and the smart scenario stored in the smart home app.

Optionally, before the mobile terminal 100 applies the corresponding smart scenario mode, a prompt message as shown in FIG. 13B will also be displayed.

As shown in FIG13B , the mobile terminal will display a window 617, which displays, for example, “It is detected that you have left home, and the “Safe Leaving Home” mode will be applied for you,” and also displays controls 617A and 617B. Control 617A is used to refuse to apply the “Safe Leaving Home” mode, and control 617B is used to agree to apply the “Safe Leaving Home” mode. When the mobile terminal does not detect the user’s action and operation of the above controls within a preset time, the mobile terminal will apply the “Safe Leaving Home” mode by default. Afterwards, the mobile terminal 100-1 will display a user interface diagram as shown in FIG13C .

As shown in FIG13C , the user interface 61 is the home page provided by the smart home APP. At this time, the device display area 614 displays the status of each home device. For example, air conditioner A, smart light A, air conditioner B, etc. have been switched to the off state.

It can be understood that the status displayed in the device display area 614 corresponds to the control rules set in the "Leave Home Safely" mode. When "Leave Home Safely" also includes turning off the TV, etc., if the TV series is on before the "Leave Home Safely" mode is applied, the mobile terminal will also control the TV to turn off, etc.

Figures 13A-13C only exemplarily show user interface diagrams of a mobile terminal controlling home devices according to a user's away-from-home status, and related interface diagrams of a mobile terminal controlling home devices according to a user's sleeping status are not described one by one here.

Next, a method is introduced in which the mobile terminal controls the home device to execute a corresponding event according to the user status and the corresponding event of the application program obtained, as follows:

The mobile terminal pre-stores the relationship between the user status and the smart scene. When the user forgets to actively turn on the corresponding smart scene, the mobile terminal can apply the corresponding smart scene mode according to the user's status. However, since the linkage control of home devices set by the smart scene is immutable, that is, different modes in the smart scene correspond to a fixed set of linkage control schemes for home devices. For example, after the sleep mode is applied, the mobile terminal will link and control the home devices to turn on the silent mode or shut down, etc., such as controlling the entertainment equipment such as TV series, audio, and projection equipment to shut down, and controlling the curtains and lights to turn off. However, when the user is sleeping, online meeting events may be recorded in the mobile terminal, but the user may miss important meetings because he turns on the sleep mode. In order to avoid this situation, the mobile terminal will also control the home lights to turn on, laptops and other office equipment in advance according to online meeting events, and remind the user that there will be online meeting events to be completed, so as to prepare for the user. Satisfy the user's home office experience. Further improve the quality of family life.

13D to 13J , which exemplarily illustrate schematic diagrams of user interfaces in which a mobile terminal controls home devices according to a user status and acquired event information.

FIG13D exemplarily shows a user interface diagram provided by the smart home app.

As shown in FIG13D , the user interface 61 is the homepage provided by the smart home APP. At this time, the device display area 614 displays the status of each home device, for example, electric curtains A, smart lights A, electric windows B, etc. are in the on state.

When the mobile terminal 100 - 1 detects that the user status is sleeping, the mobile terminal 100 - 1 will apply the smart scenario corresponding to sleep according to the correspondence between the user status and the smart scenario stored in the smart home app.

Optionally, before the mobile terminal 100 applies the corresponding smart scenario mode, a prompt message as shown in FIG. 13E will also be displayed.

As shown in FIG13E , the mobile terminal will display a window 618, which displays, for example, “It is detected that you are about to go to sleep, and the “Dream at Dawn” mode will be applied for you soon”, and also displays controls 618A and 618B. Control 618A is used to refuse to apply the “Dream at Dawn” mode, and control 618B is used to agree to apply the “Dream at Dawn” mode. When the mobile terminal does not detect the user’s action and operation of the above controls within a preset time, the mobile terminal will apply the “Dream at Dawn” mode by default. Afterwards, the mobile terminal 100-1 will display a user interface diagram as shown in FIG13F .

As shown in FIG13F , the user interface 61 is the home page provided by the smart home APP. At this time, the device display area 614 displays the status of various household devices, for example, electric curtains A, smart lights A, etc. have been switched to the off state.

It can be understood that the status displayed in the device display area 614 corresponds to the control rules set in "A Dream at Dawn". When "A Dream at Dawn" also includes turning off the TV, etc., if the TV series is on before applying the "Leave Home Safely" mode, the mobile terminal will also control the TV to turn off, etc.

Assuming that the user starts to sleep at 12:00, the mobile terminal also applies the "One Dream Daybreak" mode from 12:00. If the mobile terminal obtains event information from the weather forecast application that there will be a typhoon at 13:20, although the "One Dream Daybreak" mode does not include controlling the closing of the electric window, the mobile terminal will control the closing of the electric window B in conjunction with the event. The optional mobile terminal 100-1 will first display the prompt information shown in FIG13G.

As shown in FIG13G , a card 1218 is displayed in the user interface 121. The card 1218 is a card provided by the smart assistant installed in the mobile terminal. Specifically, the card 1218 includes a "Xiaoyi" smart assistant icon 1218A, a card reminder date 1218B, an event theme 1218C, and a sub-card 1218D.

The specific event topic 1218C is: Rainfall accompanied by typhoon is detected at 13:20 today, it is recommended that you close the doors and windows!

Sub-card 1218D mainly displays the status of smart window B. The user can click on the sub-card 1218D to jump to the device details page provided by the smart home APP and then control the device.

It is worth noting that when the mobile terminal 100-1 obtains the event information of typhoon weather at 13:00 and displays the card shown in FIG13G, if no operation for closing the smart window B is detected within the preset time, the mobile terminal 100-1 will automatically control the smart window B to close. See FIG13H for details.

As shown in FIG13H , at this time, for example, it is 13:00, and in 10 minutes, the typhoon will come. At this time, the mobile terminal 100-1 will control the electric window B to close, and the state of the electric window B in the device display area 614 has been switched to closed. In this way, even if the user is sleeping, the typhoon will not affect or damage the home environment.

If the mobile terminal learns that the user has an online meeting to attend at 14:30, and the user is still sleeping at 14:20, although the "dream to dawn" mode does not include controlling the lights in the study, laptops, etc., the mobile terminal will control the lights in the study to turn on, the laptop to start, and the smart light A and electric curtain A in the user's bedroom to turn on according to the online meeting time. The optional mobile terminal 100-1 will first display the prompt information shown in FIG13I.

As shown in FIG13I , a card 1219 is displayed in the user interface 121. The card 1219 is a card provided by the smart assistant installed in the mobile terminal. Specifically, the card 1219 includes a “Xiaoyi” smart assistant icon 1219A, a card reminder date 1219B, an event theme 1219C, and a sub-card 1219D.

The specific event topic 1219C is: An online meeting was detected at 14:30 today, and it is recommended to make relevant preparations!

Sub-card 1219D mainly displays the status of the notebook. The user can click on the sub-card 1219D to jump to the device details page provided by the smart home APP and then control the device.

It is worth noting that, after the mobile terminal 100 - 1 displays the card shown in FIG. 13I at 14:10, if no operation for turning on the notebook is detected within a preset time, the mobile terminal 100 - 1 will automatically control the notebook to turn on.

FIG. 13I merely exemplarily shows a card for prompting the user to turn on the notebook. In other embodiments of the present application, the mobile terminal 100 - 1 may also display prompt information such as turning on the study light.

As shown in FIG13J , for example, it is 14:20 at this time, and the online meeting will start in 10 minutes. At this time, the status of each household device is displayed in the device display area 614, such as the electric curtain A, the smart light A, etc. are switched to the on state again. It is worth noting that the mobile terminal 100 also controls the light in the study to turn on, and starts the notebook in the small middle, etc., which are not shown in FIG13J for the time being.

The event information of participating in the online meeting at 14:30 recorded by the above-mentioned mobile terminal can be called the first event. 14:30 belongs to the sixth time period. When the mobile terminal detects that the user will execute the first event in the sixth time period, the mobile terminal can execute the twelfth operation, which is, for example, as shown above: used to control the second device, such as a laptop to start up.

In other embodiments of the present application, after the mobile terminal controls the second device, such as a laptop to start up, the mobile terminal will also display a third control, which is used to withdraw the previous operation, that is, to withdraw the operation of controlling the laptop to start up. The mobile terminal detects the operation acting on the third control and can control the laptop to shut down.

Finally, the mobile terminal controls the home device to execute the corresponding event only according to the event corresponding to the application program obtained, which specifically includes:

The smart home APP installed in the mobile terminal can obtain the recorded events of the application. The types of application and event have been introduced above and will not be repeated here. Here, only the method of controlling home devices according to the event information of the mobile terminal is described in detail.

Here, we first take the example of the mobile terminal 100 obtaining information about online meeting events recorded in the application as shown in Figure 12M above. When the mobile terminal obtains that the user will attend an online meeting at 14:30 on the afternoon of September 26, the mobile terminal will control the light in the study to turn on before the meeting starts, and control the startup of office equipment such as laptops, etc.

Referring to FIG. 13K-FIG. 13L, FIG. 13K-FIG. 13L exemplarily illustrate a method in which a mobile terminal controls a home device to execute a corresponding event only according to an acquired event, as follows:

When mobile terminal 100 - 1 obtains that Xiaozhong has reserved a table at Restaurant A at 17:30 on a life service APP, mobile terminal 100 will control Xiaozhong's car to start according to Xiaozhong's travel habits, for example, Xiaozhong often travels by car.

FIG. 13K exemplarily shows a schematic diagram of a user interface displayed by a mobile terminal for prompting a user of pending events.

As shown in FIG13K , a card is displayed in the user interface 121 to prompt the user to start the vehicle and go to restaurant A. The event subject displayed in the card is: Restaurant A with your reservation at 17:30 is detected. It is recommended that you leave as soon as possible! In addition, the card also displays the current status of the vehicle, for example, the vehicle is currently locked.

It is worth noting that, after the mobile terminal 100 - 1 displays the card shown in FIG. 13K at 17:00, if no operation for starting the vehicle is detected within a preset time, the mobile terminal 100 - 1 will automatically control the vehicle to start.

As shown in FIG13L, for example, it is 17:10 at this time, and it will be lunch time in 20 minutes. At this time, the mobile terminal 100-1 will automatically control the vehicle to start, and display the card prompt information shown in FIG13L. The specific event displayed in the card is the theme: The vehicle has been started for you. If you do not get in the vehicle within 10 minutes, it will automatically shut down. And the status of the vehicle displayed in the sub-card is the start state.

The event information of dining out at 17:00 recorded by the above mobile terminal can be called the first event. 17:00 belongs to the sixth time period. When the mobile terminal detects that the user will perform the first event within the sixth time period, the mobile terminal can perform the twelfth operation, which is output as above: used to control the start of the vehicle.

In other embodiments of the present application, after the mobile terminal controls the vehicle to start, the mobile terminal will also display a third control, which is used to withdraw the previous operation, that is, to withdraw the vehicle start operation. The mobile terminal detects the operation acting on the third control and can control the vehicle to shut down.

It is understandable that the above Figures 13K-13L only take the event of a user making a dining reservation as an example to introduce the user interface diagram of a mobile terminal controlling home devices based only on event information, and should not constitute a limitation to the present application.

In other embodiments of the present application, the mobile terminal 100-1 can also control home devices based on other events. For example, based on the acquired event of the user exercising at home, the corresponding equipment in the home gym can be controlled to start up. These are not elaborated here.

In some other embodiments of the present application, when the eleventh operation input by the user is detected, the vehicle can lock the vehicle and lock the doors. That is to say, after the user gets off the vehicle and locks the vehicle, if the vehicle detects that there are children or elderly people in the vehicle, the vehicle's onboard computer will output the ninth prompt information for picking up children or elderly people in the vehicle, or the vehicle will also trigger the mobile terminal or other home devices to output the ninth prompt information for reminding the user that there are users in the vehicle. In addition, in order to ensure the life safety of children and the elderly, the vehicle may automatically control the opening of doors and windows. Send the eighth information to the user's mobile terminal to remind the user that there are elderly people and children in the car, the doors and windows are already open, please do not lock the doors and windows, avoid dangerous events, affect the life safety of children and the elderly, and then remind the user to take safety measures in time. Afterwards, the user's mobile terminal can also display a second control for locking windows and doors, and the user can control the vehicle to re-lock the windows and doors by acting on the control.

The detection of user status in the mobile terminal 100-1 introduced in the above Figures 12A to 12Z is related to the collection of user data introduced in the previous data bank system, which is specifically manifested in that home devices such as smart door locks, home monitoring, etc. collect the user status; or smart home applications collect the user status; or the OS of the mobile terminal collects user data such as relevant data of disconnecting the network connection with home devices; or the information input device of the mobile terminal 100-1 such as a display screen collects user data.

The mobile terminal 100-1 introduced in Figures 13A-13L above controls the home devices according to the user status, that is, it is related to the processing, opening and presentation of the user data introduced in the data bank system in the previous article. Specifically, the mobile terminal processes the collected user travel event information into the corresponding user away from home status, and then controls the home devices to perform corresponding operations. The home devices open the execution result-related data to the mobile terminal, and the mobile terminal presents the value of the execution result-related data to the user, which is specifically manifested in the prompt information that the home device has completed the corresponding operation, or the status information of the device presented in the smart home APP changes, etc.

The methods of collecting, processing, opening up, and presenting the value of user data will not be elaborated here. For details, please refer to the introduction to the data bank system in the previous article.

The method for obtaining one or more of the user's travel data, habit data, and device data by the Xiaoyi Suggestion APP, Smart Home APP, Life Service APP, Event Memo APP, etc. mentioned in the embodiments of this application can refer to the implementation method of the Smart Home APP obtaining user identity information, home space information, and device usage rules in the aforementioned embodiments. No further details will be given here.

The APPs mentioned in this application, such as Xiaoyi Suggestions APP, Smart Home APP, Life Service APP, Event Memo APP, etc., are not limited to being APPs, but can also be mini-programs, quick applications, web applications, and other programs that can be run on electronic devices.

The user interface described in the embodiments of the present application is for illustrative purposes only and should not be construed as limiting the present application.

It can be seen that after implementing the control method of the home device provided by the mobile terminal, the mobile terminal can set the user's authority to control the home device according to factors such as the role of the user in the family relationship and the space of the user in the home space. Afterwards, the mobile terminal receives the user's operation for controlling the home device, identifies the user's identity, and controls the home device to perform the corresponding function according to the authority corresponding to the user's identity.

Furthermore, parents can also set rules for their children on how to use home devices, as well as punishments for breaking the rules, thereby saving parents time in educating their children and helping them develop good living and learning habits.

Furthermore, the mobile terminal can automatically control home devices to perform corresponding operations based on the user status and/or event information recorded in the user terminal device, thereby preventing the user from neglecting to control home devices due to forgetfulness or absence from home, which may lead to the occurrence of dangerous events.

The various implementation modes of the present application can be combined arbitrarily to achieve different technical effects.

In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in this application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integration. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk), etc.

A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by a computer program to instruct the relevant hardware, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above-mentioned method embodiments. The aforementioned storage medium includes: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

In short, the above description is only an embodiment of the technical solution of the present invention, and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made according to the disclosure of the present invention shall be included in the protection scope of the present invention.

Claims (24)

1. A method for controlling a device, the method being applied to a communication system comprising a first device and a second device; wherein the second device is located in a first region in the first space; the method comprises the following steps:

the first device receives a first operation;

the first equipment identifies the identity of the user who inputs the first operation as a first user;

the first equipment triggers the second equipment to start a first function;

The first device receives the first operation again;

the first equipment identifies the identity of the user who inputs the first operation as a second user;

the first device refuses to trigger the second device to start the first function.

2. The method of claim 1, wherein the communication system further comprises a third device; before the first device receives the first operation, the method further comprises:

the first device determines that the second device is located in the first area in the first space after receiving a second operation;

or,

the first device receives first information sent by the third device, wherein the first information indicates that the second device is located in the first area in the first space.

3. The method of claim 1 or 2, wherein after the first device refuses to trigger the second device to initiate the first function, the method further comprises:

the first equipment identifies the identity of the user who inputs the first operation again;

under the condition that the user has the authority to control the equipment in the first area, the first equipment triggers the second equipment to start the first function;

And under the condition that the user does not have the right to control the equipment in the first area, the first equipment refuses to trigger the second equipment to start the first function.

4. The method of any of claims 1-3, wherein prior to the first device triggering the second device to initiate the first function, the method further comprises one or more of:

the first device determines that there are other users in the first space;

or, the first device determines that the distance between the user and the first space is less than a first value;

or, the first device determines that the time for receiving the first operation is the non-working time of the user;

or, the first device determines that the first function is used to reduce household power consumption.

5. A control method of a device, the method being applied to a communication system of a fourth device and a fifth device; characterized in that the method comprises:

the fourth device detects that the fourth device is located outside the first electronic fence;

the fourth device outputs second prompt information, or the fourth device sends second information to a fifth device, wherein the second information is used for indicating that the fourth device is located outside the first electronic fence;

The first electronic fence includes a plurality of enclosed areas and routes between the plurality of enclosed areas.

6. The method according to claim 5, wherein the fourth device detecting that the fourth device is located outside the first electronic fence specifically comprises:

the fourth device detecting that the fourth device is located outside of the plurality of enclosed regions in the first electronic fence;

or,

the fourth device detects that the fourth device is located outside the route in the first electronic fence for more than a first distance and/or is located outside the route in the first electronic fence for more than a first period of time.

7. The method of claim 6, further comprising:

the fourth device detects that the fourth device is located within a second electronic fence; and the fourth device outputs third prompt information, or the fourth device sends third information to the fifth device, wherein the third information is used for indicating that the fourth device is located in the second electronic fence.

8. A control method of a device, the method being applied to a communication system of a first device, a second device, and a fourth device; characterized in that the method comprises:

The fourth equipment detects that the service condition of the fourth equipment does not accord with a first rule;

the fourth device stops providing part of or all of the functions of the fourth device;

the fourth device receives fourth information sent by the second device, wherein the fourth information indicates that a user completes a first task; in response to receiving the fourth information, the fourth device continues to provide some or all of the functionality of the fourth device;

or,

the fourth device receives fifth information sent by the first device, wherein the fifth information is used for instructing the fourth device to continue to provide part or all of functions of the fourth device; in response to receiving the fifth information, the fourth device continues to provide some or all of the functionality of the fourth device.

9. The method of claim 8, wherein the fourth device is before or after receiving the fourth information sent by the second device, and before the fourth device continues to provide part or all of the functionality of the fourth device; the method further comprises the following steps:

the second equipment sends the fourth information to the first equipment;

the first device detects a fifth operation; and the first equipment sends the fifth information to the fourth equipment, wherein the fifth information is used for indicating the fourth equipment to continuously provide part or all functions of the fourth equipment.

10. The method according to claim 8 or 9, wherein after the fourth device receives the fourth information sent by the second device, and before the fourth device continues to provide part or all of the functions of the fourth device; the method further comprises the following steps:

the fourth device sends the third information to the first device;

and the first device detects a fifth operation, and the first device sends fifth information to the fourth device, wherein the fifth information is used for instructing the fourth device to continue to provide part or all of functions of the fourth device.

11. The method according to any of claims 8-10, wherein the first rule comprises any one or more of:

the time length of the fourth device in the unlocking state every day is less than the first time length;

the duration that the fourth device is continuously in the unlocking state is less than the second duration;

the fourth equipment is in a screen locking state at other time except for the third time period;

the fourth device refuses to run the application programs except the first application program;

and the duration of the continuous running of the first application program by the fourth equipment is less than the third duration.

12. The method according to any one of claims 8 to 11, wherein the receiving, by the fourth device, the fourth information sent by the second device specifically includes:

and the second equipment authenticates the identity of the user and sends third information to the fourth equipment under the condition that the identity of the user is the second user.

13. A control method of a device is applied to a communication system of a sixth device of a first device; characterized in that the method comprises:

the first equipment detects a sixth operation, and the sixth operation is used for starting a second function;

the first equipment acquires the state of a second user according to a seventh operation input by the user, or acquires the state of the second user through sixth equipment;

in the event that the status of the second user is one or more of: when the user does not sleep or study, the distance between the second user and the first user is smaller than the second distance and the preset distance, or the second user and the first user are in a room;

the first device refuses to initiate the second function.

14. A control method of a device, the method is applied to a communication system of a first device and a sixth device; characterized in that the method comprises:

The first device detects an eighth operation input by the second user, and the eighth operation is used for starting a third function;

the first equipment acquires the state of the first user;

in the event that the status of the first user is one or more of: sleeping, leaving home, working;

the first device refuses to initiate the third function.

15. A control method of equipment is applied to a communication system comprising an intelligent door lock, first equipment and second equipment; characterized in that the method comprises:

when the intelligent door lock detects a ninth operation, the intelligent door lock opens the door;

when the distance between the intelligent door lock and the first equipment becomes larger gradually, the intelligent door lock executes one or more of the following: and refusing to close the door, and outputting fifth prompt information for prompting a user to carry the first equipment, or triggering the first equipment and/or the second equipment to output fifth prompt information for prompting the user to carry the first equipment.

16. The method of claim 15, wherein the smart door lock, the first device, and the second device are located in a first space; the method further comprises the following steps:

During a fourth time period, the first device determines that the user leaves the first space, and the user goes out with the first item during the fourth time period;

and the first equipment outputs sixth prompt information, wherein the sixth prompt information is used for prompting a user to carry the first article.

17. The method of claim 15 or 16, wherein the smart door lock, the first device, and the second device are located in a first space, the method further comprising:

during a fifth time period, the first device determines that a user leaves the first space; extreme weather occurs during the fifth time period;

and the first equipment outputs seventh prompt information, and the seventh prompt information is used for prompting a user to carry a protective article.

18. The method of any one of claims 15-17, wherein the communication system further comprises a vehicle; the method further comprises the following steps:

the vehicle detects a tenth operation, and a door of the vehicle is opened;

as the distance between the second door and the first device becomes progressively larger, the vehicle performs one or more of: and refusing to close the vehicle door, and outputting eighth prompt information for prompting a user to carry the first equipment, or triggering the first equipment and/or the second equipment to output eighth prompt information for prompting the user to carry the first equipment.

19. The method of any of claims 15-18, wherein the communication system further comprises a vehicle; the method further comprises the following steps:

the vehicle detects an eleventh operation, and the vehicle locks the door and the window;

the vehicle detecting a user within the vehicle, the vehicle performing one or more of: and opening a window and a door, and outputting ninth prompt information for prompting the user in the vehicle, or triggering the first equipment and/or the second equipment to output the ninth prompt information for prompting the user in the vehicle.

20. A control method of a device is applied to a communication system comprising a first device, a second device and a vehicle; characterized in that the method comprises:

in a sixth time period, the first device performs a twelfth operation; the user will execute the first event during the sixth time period;

the twelfth operation may include any one or more of:

the first device triggers the third device to start a fifth function, and the fifth function supports a user to execute the first event;

or the first device triggers the vehicle to start, and the vehicle starting support user executes the first event.

21. An electronic device, characterized in that the electronic device comprises one or more processors and one or more memories; wherein the one or more memories are coupled to the one or more processors for storing computer program code comprising computer instructions that, when executed by the one or more processors, cause the electronic device to perform the method of any of claims 1-3.

22. A computer program product comprising instructions for causing an electronic device to perform the method according to any one of claims 1-3 when the computer program product is run on the electronic device.

23. A computer-readable storage medium comprising instructions that, when executed on an electronic device, cause the electronic device to perform the method of any of claims 1-3.

24. A communication system, characterized in that the communication system comprises a first electronic device, a second electronic device, the first electronic device being configured to perform the method according to any of claims 1-3.

Images