KR102415820B1 - Control method, device and program of smart device system using artificial intelligence based surface recognition method - Google Patents

Abstract

A method for controlling a smart device system is provided. The method for controlling the smart device system may include: collecting, by the server, learning data for acquiring a location of the electronic device based on surface information; learning, by the server, an artificial intelligence model based on the learning data; receiving, by the server, input data for obtaining a location of the electronic device from the electronic device; obtaining, by the server, at least one of surface information and location information of the electronic device by inputting the input data into the artificial intelligence model; determining, by the server, a service to be provided by the electronic device based on at least one of the acquired surface information and location information of the electronic device; and transmitting, by the server, information on the determined service to the electronic device. includes

Description

CONTROL METHOD, DEVICE AND PROGRAM OF SMART DEVICE SYSTEM USING ARTIFICIAL INTELLIGENCE BASED SURFACE RECOGNITION METHOD

The present invention relates to a method, apparatus and program for controlling a smart device system through an artificial intelligence-based surface recognition method.

With the development of smart devices, technologies for providing various conveniences to users are being developed. In particular, based on the location of smart devices such as artificial intelligence speakers and smart phones, various methods have been proposed to provide services highly related to the location to increase user experience.

In order to provide these services, it is important to know the location of the smart device. Although various methods for determining a location have been proposed, the method for determining a location using a camera may be greatly affected by lighting conditions or may cause an issue of privacy.

Accordingly, the need for technology development for locating the location of the smart device is increasing.

Laid-Open Patent Publication No. 10-2005-0074897, 2005.07.19

An object to be solved by the present invention is to provide a method, apparatus and program for controlling a smart device system through an artificial intelligence-based surface recognition method.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

In a method for controlling a smart device system including a server and an electronic device according to an aspect of the present invention for solving the above-described problems, the server receives learning data for acquiring the location of the electronic device based on surface information collecting; learning, by the server, an artificial intelligence model based on the learning data; receiving, by the server, input data for obtaining a location of the electronic device from the electronic device; obtaining, by the server, at least one of surface information and location information of the electronic device by inputting the input data into the artificial intelligence model; determining, by the server, a service to be provided by the electronic device based on at least one of the acquired surface information and location information of the electronic device; and transmitting, by the server, information on the determined service to the electronic device. includes

In this case, the collecting of the learning data may include: generating, by the shock generating unit included in the electronic device, at least one of an internal shock and an external shock; detecting, by a sensor included in the electronic device, the generated shock; and transmitting, by the electronic device, the sensed impact data to the server. may include

In this case, the collecting of the learning data may include: obtaining, by the server, the impact data, object information corresponding to the impact data, and spatial information in which the object is located; and storing, by the server, as the learning data by matching at least two of the impact data, object information corresponding to the impact data, and spatial information in which the object is located. may include

In this case, the impact data may include at least one of inertial data acquired by the IMU sensor and audio data acquired by a microphone.

At this time, in the step of learning the artificial intelligence model, when the server acquires the first shock data generated by the internal shock as the learning data, the first shock data is input to the first artificial intelligence model to 1 training the artificial intelligence model; and when the server acquires the second shock data generated by the external shock as the learning data, inputting the second shock data into a second artificial intelligence model to learn the second artificial intelligence model; may include.

In this case, the receiving of the input data may include: generating, by the shock generating unit included in the electronic device, at least one of an internal shock and an external shock; detecting, by a sensor included in the electronic device, at least one of the generated internal shock and the generated external shock; and transmitting, by the electronic device, shock data of at least one of a third shock data for the sensed internal shock and the sensed fourth shock data to the server. may include

In this case, the step of obtaining at least one of the surface information and the location information of the electronic device may include: obtaining, by the server, the third impact data into the first artificial intelligence model as a first output value; obtaining, by the server, the fourth impact data as a second output value by inputting the fourth impact data into the second artificial intelligence model; and obtaining, by the server, at least one of surface information and location information of the electronic device based on the first output value and the second output value; may include

In this case, the obtaining of at least one of the surface information and the location information of the electronic device may include inputting the fourth impact data into the second artificial intelligence model when the third impact data is less than or equal to a preset value to the electronic device obtaining at least one of surface information and location information of and obtaining at least one of surface information and location information of the electronic device by inputting the third impact data into the first artificial intelligence model when the fourth impact data is equal to or less than a preset value. .

Other specific details of the invention are included in the detailed description and drawings.

According to the above-described various embodiments of the present invention, the electronic device obtains surface information of a place where the electronic device is located by using an internal shock or an external shock, and obtains location information based on the acquired surface information to provide various services to the user can provide

Effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a system diagram according to an embodiment of the present invention.
2A and 2B are block diagrams of a server and an electronic device according to an embodiment of the present invention.
3 is a diagram illustrating an electronic device according to an embodiment of the present invention.
4A and 4B are exemplary views for explaining an operation of an electronic device according to an embodiment of the present invention.
5 is a flowchart illustrating an operation of a system according to an embodiment of the present invention.
6 is a flowchart illustrating impact data generated by an electronic device according to an embodiment of the present invention.
7 is a flowchart illustrating a method of acquiring learning data according to an embodiment of the present invention.
8 is a flowchart illustrating a method for learning an artificial intelligence model according to an embodiment of the present invention.
9 is a flowchart illustrating a method of obtaining an output value using an artificial intelligence model according to an embodiment of the present invention.
10 is a flowchart illustrating a method of acquiring surface information according to an embodiment of the present invention.
11 is a flowchart illustrating a method of acquiring surface information according to types of internal impact and external impact according to an embodiment of the present invention.

Advantages and features of the present invention and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only the present embodiments allow the disclosure of the present invention to be complete, and those of ordinary skill in the art to which the present invention pertains. It is provided to fully understand the scope of the present invention to those skilled in the art, and the present invention is only defined by the scope of the claims.

The terminology used herein is for the purpose of describing the embodiments and is not intended to limit the present invention. In this specification, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, “comprises” and/or “comprising” does not exclude the presence or addition of one or more other components in addition to the stated components. Like reference numerals refer to like elements throughout, and "and/or" includes each and every combination of one or more of the recited elements. Although "first", "second", etc. are used to describe various elements, these elements are not limited by these terms, of course. These terms are only used to distinguish one component from another. Accordingly, it goes without saying that the first component mentioned below may be the second component within the spirit of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used herein will have the meaning commonly understood by those of ordinary skill in the art to which this invention belongs. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless specifically defined explicitly.

As used herein, the term “unit” or “module” refers to a hardware component such as software, FPGA, or ASIC, and “unit” or “module” performs certain roles. However, “part” or “module” is not meant to be limited to software or hardware. A “unit” or “module” may be configured to reside on an addressable storage medium or to reproduce one or more processors. Thus, as an example, “part” or “module” refers to components such as software components, object-oriented software components, class components and task components, processes, functions, properties, Includes procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables. Components and functionality provided within “parts” or “modules” may be combined into a smaller number of components and “parts” or “modules” or as additional components and “parts” or “modules”. can be further separated.

Spatially relative terms "below", "beneath", "lower", "above", "upper", etc. It can be used to easily describe the correlation between a component and other components. Spatially relative terms should be understood as terms including different directions of components during use or operation in addition to the directions shown in the drawings. For example, when a component shown in the drawing is turned over, a component described as “beneath” or “beneath” of another component may be placed “above” of the other component. can Accordingly, the exemplary term “below” may include both directions below and above. Components may also be oriented in other orientations, and thus spatially relative terms may be interpreted according to orientation.

In this specification, a computer means all types of hardware devices including at least one processor, and may be understood as encompassing software configurations operating in the corresponding hardware device according to embodiments. For example, a computer may be understood to include, but is not limited to, smart phones, tablet PCs, desktops, notebooks, and user clients and applications running on each device.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Each step described in this specification is described as being performed by a computer, but the subject of each step is not limited thereto, and at least a portion of each step may be performed in different devices according to embodiments.

1 is a system diagram according to an embodiment of the present invention.

1 , the smart device control system may include a server 100 and an electronic device 200 .

The server 100 according to an embodiment of the present invention may be configured not only as a single server, but also as a plurality of servers, if necessary, and may be a concept including a cloud server.

The electronic device 200 may generate impact data for obtaining surface information. However, it goes without saying that this only represents one function of the electronic device 200 and that various services may be provided to the user.

In an embodiment, the electronic device 200 includes a smartphone, a tablet personal computer, a mobile phone, a video phone, an e-book reader, and a desktop PC. ), laptop PC, netbook computer, workstation, server, personal digital assistant (PDA), portable multimedia player (PMP), MP3 player, mobile medical device, camera, or wearable device (wearable device) and may include at least one of an artificial intelligence speaker (AI speaker).

As another embodiment, the electronic device 200 may be a home appliance. Home appliances are, for example, televisions, DVD players (Digital Video Disk players), audio systems, refrigerators, air conditioners, vacuum cleaners, ovens, microwaves, washing machines, air purifiers, set-top boxes, and home automation. Control panel (home automation control panel), security control panel (security control panel), TV box (eg Samsung HomeSync™, Apple TV™, or Google TV™), game console (eg Xbox™, PlayStation™), electronics It may include at least one of a dictionary, an electronic key, a camcorder, and an electronic picture frame.

Although the electronic device 200 has a configuration capable of generating an impact as an essential component, it goes without saying that it may include a configuration for executing various other functions. For example, the electronic device 200 may be configured to include a shock generating unit inside the AI speaker. Alternatively, the electronic device may include only a shock generating unit and may be in the form of an accessory that can be coupled to a smart phone or the like.

2A and 2B are block diagrams of a server and an electronic device according to an embodiment of the present invention.

As shown in FIG. 2A , the server 100 may include a memory 110 , a communication unit 120 , and a processor 130 .

The memory 110 may store various programs and data necessary for the operation of the server 100 . The memory 110 may be implemented as a non-volatile memory 110 , a volatile memory 110 , a flash-memory 110 , a hard disk drive (HDD), or a solid state drive (SSD).

The communication unit 120 may communicate with an external device. In particular, the communication unit 120 may include various communication chips such as a Wi-Fi chip, a Bluetooth chip, a wireless communication chip, an NFC chip, and a Bluetooth low power chip (BLE chip). At this time, the Wi-Fi chip, the Bluetooth chip, and the NFC chip perform communication in a LAN method, a WiFi method, a Bluetooth method, and an NFC method, respectively. In the case of using a Wi-Fi chip or a Bluetooth chip, various types of connection information such as an SSID and a session key are first transmitted and received, and then various types of information can be transmitted and received after a communication connection using this. The wireless communication chip refers to a chip that performs communication according to various communication standards, such as IEEE, ZigBee, 3rd Generation (3G), 3rd Generation Partnership Project (3GPP), and Long Term Evolution (LTE).

The processor 130 may control the overall operation of the server 100 using various programs stored in the memory 110 . The processor 130 may include a RAM, a ROM, a graphic processing unit, a main CPU, first to n interfaces, and a bus. In this case, the RAM, ROM, graphic processing unit, main CPU, first to n interfaces, etc. may be connected to each other through a bus.

RAM stores O/S and application programs. Specifically, when the server 100 is booted, O/S may be stored in the RAM, and various application data selected by the user may be stored in the RAM.

The ROM stores an instruction set for booting the system, and the like. When a turn-on command is input and power is supplied, the main CPU copies the O/S stored in the memory 110 to the RAM according to the command stored in the ROM, and executes the O/S to boot the system. Upon completion of booting, the main CPU copies various application programs stored in the memory 110 to the RAM, and executes the application programs copied to the RAM to perform various operations.

The graphic processing unit generates a screen including various objects such as items, images, and texts by using a calculation unit (not shown) and a rendering unit (not shown). Here, the calculation unit may be configured to calculate attribute values such as coordinate values, shape, size, color, etc. of each object to be displayed according to the layout of the screen by using the control command received from the input unit. In addition, the rendering unit may be configured to generate screens of various layouts including objects based on the attribute values calculated by the operation unit. The screen generated by the rendering unit may be displayed in the display area of the display.

The main CPU accesses the memory 110 and performs booting using the OS stored in the memory 110 . In addition, the main CPU performs various operations using various programs, contents, data, etc. stored in the memory 110 .

The first to n interfaces are connected to the various components described above. One of the first to n interfaces may be a network interface connected to an external device through a network.

Meanwhile, further, the processor 130 may control the artificial intelligence model. In this case, of course, the processor 130 may include a graphics-only processor (eg, GPU) for controlling the artificial intelligence model.

Meanwhile, the artificial intelligence model according to the present invention may be a model based on supervised learning or unsupervised learning by a teacher. Furthermore, the artificial intelligence model according to the present invention may include a support vector machine (SVM), a decision tree, a neural network, and the like, and methodologies to which these are applied.

In one embodiment, the artificial intelligence model according to the present invention may be an artificial intelligence model based on a convolutional deep neural network (CNN) learned by inputting learning data. However, the present invention is not limited thereto, and it goes without saying that various artificial intelligence models may be applied to the present invention. For example, a model such as a Deep Neural Network (DNN), a Recurrent Neural Network (RNN), or a Bidirectional Recurrent Deep Neural Network (BRDNN) may be used as the AI model, but is not limited thereto.

In this case, convolutional deep neural networks (CNNs) are a type of multilayer perceptrons designed to use minimal preprocessing. A convolutional neural network consists of one or several convolutional layers and general artificial neural network layers on top of it, and additionally utilizes weights and pooling layers. Thanks to this structure, the convolutional neural network can fully utilize the input data of the two-dimensional structure. In addition, convolutional neural networks can be trained via standard backpropagation. Convolutional neural networks are easier to train than other feed-forward neural network techniques and have the advantage of using fewer parameters.

In addition, deep neural networks (DNNs) are artificial neural networks (ANNs) composed of a plurality of hidden layers between an input layer and an output layer.

In this case, the structure of the deep neural network may be composed of a perceptron. A perceptron consists of several inputs, one processor, and one output value. The processor multiplies each input value by a weight, and then sums the input values multiplied by the weight. Then, the processor outputs one output value by substituting the summed value into the activation function. If a specific value is desired as the output value of the activation function, the weight multiplied by each input value may be corrected, and the output value may be recalculated using the modified weight. In this case, each perceptron may use a different activation function. In addition, each perceptron receives the outputs from the previous layer as input, and then uses the activation function to obtain the output. The obtained output is transferred to the input of the next layer. Through the process as described above, some output values can be finally obtained.

A recursive neural network (RNN) refers to a neural network in which connections between units constituting an artificial neural network constitute a directed cycle. Unlike forward neural networks, recurrent neural networks can utilize the memory inside the neural network to process arbitrary inputs.

Deep Belief Networks (DBN) is a generative graphical model used in machine learning, and in deep learning, it means a deep neural network composed of multiple layers of latent variables. There is a connection between layers, but there is no connection between units within a layer.

The deep trust neural network can be used for prior learning due to the nature of the generative model, and after learning the initial weights through prior learning, the weights can be fine-tuned through backpropagation or other discrimination algorithms. This characteristic is very useful when the training data is small, because the smaller the training data, the stronger the effect of the initial value of the weight on the resulting model. The pre-learned initial weight value is closer to the optimal weight compared to the arbitrarily set initial weight value, which enables performance and speed improvement in the fine-tuning stage.

The above-described artificial intelligence and its learning method have been described for illustrative purposes, and the artificial intelligence and its learning method used in the embodiments described below are not limited. For example, all kinds of artificial intelligence technology and a learning method thereof that a person skilled in the art can apply to solve the same problem may be utilized to implement the system according to the disclosed embodiment.

Meanwhile, as shown in FIG. 2B , the electronic device 200 includes at least one sensor 210 , an input unit 220 , an output unit 230 , a memory 240 , a shock generator 250 , and a processor. can do.

At least one sensor 210 is configured to detect an impact generated by the impact generating unit 250 . In an embodiment, the at least one sensor 210 may include an Inertial Measurement Unit (IMU) sensor. Specifically, the at least one sensor 210 may include at least one sensor among a gyroscope, an acceleration sensor, and a geomagnetic sensor.

In another embodiment, the at least one sensor 210 may include a microphone for receiving sound data (or audio data). The microphone may receive an audio signal generated by the impact.

The input unit 220 is configured to receive a user command. In this case, the input unit 230 may be configured to receive various types of user commands, such as a button, a keyboard, a mouse, and a touch panel.

The output unit 230 is configured to output various types of data, such as an audio image. The output unit 230 according to the present invention may include at least one of a speaker and a display.

The memory 240 may store various programs and data necessary for the operation of the electronic device 200 , and may include the above-described configuration of the memory 110 .

The impact generating unit 250 is configured to generate an impact. Specifically, the impact generating unit 250 may include a housing 251 and an impact module 252 as shown in FIG. 3 . The impact module 252 may be implemented as an actuator for vertical movement (eg, an actuator using a solenoid).

The processor 260 is a configuration for controlling the electronic device 200 and may include the configuration of the processor 130 described above.

Meanwhile, according to various embodiments of the present disclosure, when a preset condition is satisfied, the electronic device 200 may transmit a control signal to operate the shock generating unit 250 . For example, the electronic device 200 may detect the movement of the electronic device 200 and transmit a control signal to operate the shock generator 250 when the movement is stopped.

Alternatively, the electronic device 200 may transmit a control signal to operate the shock generating unit 250 when the electronic device 200 is placed on an object. For example, when detecting an impact greater than or equal to a preset value, the electronic device 200 may determine that movement to another location has been completed, and transmit a control signal to operate the impact generator 250 .

4A and 4B are exemplary views for explaining an operation of an electronic device according to an embodiment of the present invention.

Specifically, as described above, the shock module 252 may move up and down, and when the shock module 252 moves in the upward direction, an impact may be applied to the inside of the electronic device 200, and the shock module 252 ( When the 252 ) moves in the downward direction, the electronic device 200 may apply an impact to the surface of the external object in contact with it.

In this case, the internal shock data generated by the internal shock and the external shock data generated by the external shock may be different data. For example, the mechanical impulse response of the internal shock data generated by the internal shock and the mechanical impulse response of the external shock data generated by the external shock may be different. Alternatively, the acoustic data generated by the internal impact and the acoustic data generated by the external impact may be different.

Accordingly, the server 100 may distinguish the internal impact data from the external impact data, and may acquire information on the surface in contact with the electronic device 200 according to each characteristic. However, of course, the server 100 may not only use both internal shock data and external shock data, but also obtain surface information using at least one of internal shock data and external shock data.

Hereinafter, various embodiments according to the present invention will be described in detail with reference to FIGS. 5 to 11 .

5 is a flowchart illustrating an operation of a system according to an embodiment of the present invention.

In S110 , the server 100 may collect learning data for acquiring the location of the electronic device 200 based on the surface information.

As an embodiment, the server 100 may obtain impact data including at least one of inertial data and acoustic data related to surface learning from the electronic device 200 . In this case, the inertia data may include data about acceleration or data about angular velocity.

However, according to an embodiment of the present invention, of course, the server 100 may use only some data of the various impact data described above. That is, the impact data may include at least one of acceleration data, angular velocity data, and an acoustic signal.

Furthermore, the server 100 may acquire the acquired impact data and surface information corresponding to the acquired impact data as class information.

That is, the server 100 may acquire the impact data and the surface information data corresponding to the impact data as learning data for learning the artificial intelligence model by matching it.

Furthermore, of course, the server 100 may additionally acquire location information corresponding to the surface information data, and obtain it as class information.

In S120, the server 100 may train an artificial intelligence model based on the learning data.

As an embodiment, the server 100 may model and store the surface characteristics in the form of machine learning parameters based on the learning data.

In S130 , the server 100 may receive input data for obtaining a location of the electronic device 200 from the electronic device 200 .

In one embodiment, the input data may be the various impact data described above. That is, the input data may be internal shock data or external shock data. Alternatively, the input data may be inertial data or acoustic data for impact.

In S140 , the server 100 may obtain at least one of surface information and location information of the electronic device 200 by inputting the input data into the artificial intelligence model.

In S150 , the server 100 may determine a service to be provided by the electronic device 200 based on at least one of the acquired surface information and location information of the electronic device 200 .

In S160 , the server 100 may transmit information on the determined service to the electronic device 200 .

6 is a flowchart illustrating impact data generated by an electronic device according to an embodiment of the present invention.

In S210 , the shock generating unit included in the electronic device 200 may generate at least one of an internal shock and an external shock.

In S220 , at least one sensor 210 included in the electronic device 200 may detect the generated shock.

In S230 , the electronic device 200 may transmit the sensed shock data to the server 100 .

7 is a flowchart illustrating a method of acquiring learning data according to an embodiment of the present invention.

In S310, the server 100 may acquire impact data, object information corresponding to the impact data, and spatial information in which the object is located.

That is, the server 100 may store impact data, object information, and spatial information where the object is located as one learning data set.

In this case, the object information may be barcode information on the object or information on a serial number. When the object information is barcode information or serial number information for the object, the server 100 may receive impact data on the same object from another electronic device other than the electronic device 200 and set it as learning data. Through the above-described method, since the server 100 can collect learning data from a plurality of electronic devices, it is possible to perform machine learning with high accuracy.

As another embodiment, the object information may be information about an object input by the user of the electronic device 200 . For example, when the server 100 receives the impact data, it may transmit a query message inquiring about an object corresponding to the corresponding impact data to the electronic device 200 . The electronic device 200 may receive information about an object (eg, a name of an object such as a table or sofa) from the user. through the method described above. The server 100 may provide a personalized service by acquiring data suitable for a user's use environment of the electronic device 200 as learning data.

In S320 , the server 100 may match at least two of the impact data, object information corresponding to the impact data, and spatial information in which the object is located, and store it as learning data.

In one embodiment, as described above, when the object information is barcode information or serial number information for a corresponding object, the server 100 may match the impact data and the object information and store it as learning data. That is, if the object information is barcode information or serial number for the object, even if it is the same object, the object may be placed in a different location depending on the user, so it may not be desirable to include the location information of the object in the learning data. . In this case, the server 100 obtains the first learning data by matching the impact data and the object information, and obtains the second learning data by matching the information about the user of the electronic device 200 and the position information of the object. have.

As another embodiment, as described above, when the object information is information about an object input by the user of the electronic device 200 , the server 100 matches all of the impact data, the object information, and the location information of the object. can be saved as training data. That is, since the learning data in the present embodiment is data about an individual user of the electronic device 200, it is preferable to include the location information of the object in the learning data.

8 is a flowchart illustrating a method for learning an artificial intelligence model according to an embodiment of the present invention.

In S410, the server 100 may learn the first artificial intelligence model by inputting the first shock data to the first artificial intelligence model when acquiring the first shock data generated by the internal shock as learning data.

In S420, the server 100 may learn the second artificial intelligence model by inputting the second shock data to the second artificial intelligence model when acquiring the second shock data generated by the external shock as the learning data.

That is, the server 100 may learn the respective artificial intelligence models by inputting the first shock data for the internal shock and the second shock data for the external shock to the independent artificial intelligence models, respectively.

Even if this is the impact data generated by the electronic device 200 on the same object, the server 100 has different characteristics of the first impact data and the second impact data, so the first impact data and the second impact data are applied to different artificial intelligence models. The first artificial intelligence model and the second artificial intelligence model can be trained by inputting the two impact data, respectively.

9 is a flowchart illustrating a method of obtaining an output value using an artificial intelligence model according to an embodiment of the present invention.

In S510 , the shock generating unit 250 included in the electronic device 200 may generate at least one of an internal shock and an external shock.

In S520 , a sensor included in the electronic device 200 may detect at least one of a generated internal shock and a generated external shock.

In S530 , the electronic device 200 may transmit shock data of at least one of the third shock data for the sensed internal shock and the sensed fourth shock data to the server 100 .

The above-described embodiment of FIG. 9 may be the same as the method of obtaining the first impact data and the second impact data to obtain the learning data.

10 is a flowchart illustrating a method of acquiring surface information according to an embodiment of the present invention.

In S610, the server 100 may input the third impact data into the first artificial intelligence model and obtain it as a first output value.

As an embodiment, the first output value may be information on the surface of the object or information on the object obtained based on the surface information of the object. As another embodiment, the first output value may be position information of the object obtained based on information on the surface of the object.

In S620, the server 100 may input the fourth impact data to the second artificial intelligence model and obtain it as a second output value.

As an embodiment, the second output value may be surface information of the object or information about the object obtained based on the surface information of the object. As another embodiment, the second output value may be position information of the object obtained based on information on the surface of the object.

In S630 , the server 100 may acquire at least one of surface information and location information of the electronic device 200 based on the first output value and the second output value.

As an embodiment, when the object information corresponding to the first output value and the object information corresponding to the second output value are the same, the server 100 may obtain the same object information as information about the object.

However, when the object information corresponding to the first output value and the object information corresponding to the second output value are different from each other, the server 100 selects any one of the object information corresponding to the first output value and the object information corresponding to the second output value. information about can be obtained.

Specifically, the server 100 compares the first reliability value of the first output value output by the first artificial intelligence model with the second reliability value of the second output value output by the second artificial intelligence model, and the reliability value is Object information corresponding to a high output value may be acquired as information about the object.

In this case, the reliability value refers to coordinates on a vector space corresponding to an output value (eg, data on surface information) output by the artificial intelligence model, and on a vector space corresponding to data on surface information of object information obtained from the output value. It can be obtained based on the vector distance from the coordinates. As an embodiment, as the distance increases, the reliability value may decrease, and as the distance increases, the reliability value may increase.

11 is a flowchart illustrating a method of acquiring surface information according to types of internal impact and external impact according to an embodiment of the present invention.

In S710 , when the third impact data is equal to or less than a preset value, at least one of surface information and location information of the electronic device 200 may be obtained by inputting the fourth impact data into the second artificial intelligence model.

In S720 , when the fourth impact data is equal to or less than a preset value, at least one of surface information and location information of the electronic device 200 may be obtained by inputting the third impact data into the first artificial intelligence model.

That is, depending on the type of object on which the electronic device 200 is placed, obtaining surface information using an external impact has higher accuracy than obtaining surface information using an internal impact, or vice versa. can

For example, when the electronic device 200 is placed on a sofa or bed that absorbs shock well, the value of impact data due to an external impact may be very small. Therefore, the server 100 ignores the output value output by the second artificial intelligence model when the fourth shock data due to the external shock is less than or equal to the preset value, and based on the output value output by the first artificial intelligence model Information about the object can be obtained.

Conversely, the server 100 ignores the output value output by the first artificial intelligence model when the third shock data due to the internal shock is less than or equal to the preset value, and based on the output value output by the second artificial intelligence model Information about the object can be obtained. Alternatively, the server 100 ignores the output value output by the first artificial intelligence model when the difference between the value of the third shock data due to the internal shock and the value of the fourth shock data due to the external shock is less than or equal to a preset value And, based on the output value output by the second artificial intelligence model, it is possible to obtain information about the object.

Meanwhile, based on the location information of the electronic device 200 obtained according to various embodiments of the present disclosure, the server 100 may determine a service to be performed by the electronic device and control the electronic device 200 .

In an embodiment, when the object information obtained by the server 100 is table information and the location information corresponding to the table is a kitchen, the server 100 controls the electronic device 200 to provide food-related information. can do.

In another embodiment, when the information on the object obtained by the server 100 is desk information and the location information corresponding to the desk is a reading room, the server 100 controls the volume or lighting brightness of the electronic device 200 to adjust the A control signal can be transmitted.

On the other hand, the steps of the method or algorithm described in relation to the embodiment of the present invention may be implemented as hardware directly, implemented as a software module executed by hardware, or a combination thereof. A software module may contain random access memory (RAM), read only memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, hard disk, removable disk, CD-ROM, or It may reside in any type of computer-readable recording medium well known in the art to which the present invention pertains.

The components of the present invention may be implemented as a program (or application) to be executed in combination with a computer, which is hardware, and stored in a medium. Components of the present invention may be implemented as software programming or software components, and similarly, embodiments may include various algorithms implemented as data structures, processes, routines, or combinations of other programming constructs, including C, C++ , Java, assembler, etc. may be implemented in a programming or scripting language. Functional aspects may be implemented in an algorithm running on one or more processors.

In the above, embodiments of the present invention have been described with reference to the accompanying drawings, but those of ordinary skill in the art to which the present invention pertains can realize that the present invention can be embodied in other specific forms without changing the technical spirit or essential features thereof. you will be able to understand Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive.

100 : server
200: electronic device

Claims (11)

A method for controlling a smart device system including a server and an electronic device, the method comprising:
collecting, by the server, learning data for acquiring a location of the electronic device based on the impact data received from the electronic device;
learning, by the server, an artificial intelligence model based on the learning data;
receiving, by the server, input data for obtaining a location of the electronic device from the electronic device;
obtaining, by the server, the location information of the electronic device by inputting the input data into the artificial intelligence model;
determining, by the server, a service matching the location information of the electronic device to be provided by the electronic device based on the obtained location information of the electronic device; and
transmitting, by the server, information on the determined service to the electronic device; including,
The step of collecting the learning data is,
generating, by the shock generating unit included in the electronic device, at least one of an internal shock and an external shock;
detecting, by a sensor included in the electronic device, the generated shock;
transmitting, by the electronic device, the sensed shock data, which is a mechanical impulse response or acoustic data corresponding to the shock, to the server;
obtaining, by the server, the impact data, object information corresponding to the impact data, and spatial information in which the object is located; and
storing, by the server, as the learning data by matching at least two of the impact data, object information corresponding to the impact data, and spatial information in which the object is located; includes,
Acquiring the impact data, object information corresponding to the impact data, and spatial information in which the object is located includes:
Distinguish between internal shock data and external shock data;
The control method, characterized in that the object information is barcode information or serial number information on the object, or information on the object input by a user of the electronic device. delete delete According to claim 1,
The impact data is
A control method comprising at least one of inertial data acquired by an IMU sensor and audio data acquired by a microphone. 5. The method of claim 4,
The step of learning the artificial intelligence model is,
when the server acquires the first shock data generated by the internal shock as the learning data, inputting the first shock data into a first artificial intelligence model to learn the first artificial intelligence model; and
when the server acquires the second shock data generated by the external shock as the learning data, inputting the second shock data into a second artificial intelligence model to learn the second artificial intelligence model; Control method comprising a. 6. The method of claim 5,
Receiving the input data comprises:
generating, by the shock generating unit included in the electronic device, at least one of an internal shock and an external shock;
detecting, by a sensor included in the electronic device, at least one of the generated internal shock and the generated external shock; and
transmitting, by the electronic device, shock data of at least one of a third shock data for the sensed internal shock and the sensed fourth shock data to the server; A control method comprising a. 7. The method of claim 6,
Acquiring the location information of the electronic device includes:
obtaining, by the server, the third impact data as a first output value by inputting the third impact data into the first artificial intelligence model;
obtaining, by the server, the fourth impact data as a second output value by inputting the fourth impact data into the second artificial intelligence model; and
obtaining, by the server, location information of the electronic device based on the first output value and the second output value; A control method comprising a. 7. The method of claim 6,
Acquiring the location information of the electronic device includes:
when the third impact data is less than or equal to a preset value, inputting the fourth impact data into the second artificial intelligence model to obtain location information of the electronic device; and
When the fourth impact data is less than or equal to a preset value, inputting the third impact data into the first artificial intelligence model to obtain location information of the electronic device; delete delete delete

Images