CN107731020B - Multimedia playing method, device, storage medium and electronic equipment - Google Patents
Multimedia playing method, device, storage medium and electronic equipment Download PDFInfo
- Publication number
- CN107731020B CN107731020B CN201711085728.9A CN201711085728A CN107731020B CN 107731020 B CN107731020 B CN 107731020B CN 201711085728 A CN201711085728 A CN 201711085728A CN 107731020 B CN107731020 B CN 107731020B
- Authority
- CN
- China
- Prior art keywords
- image
- multimedia file
- preview frame
- module
- preview
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Classifications
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B5/00—Electrically-operated educational appliances
- G09B5/04—Electrically-operated educational appliances with audible presentation of the material to be studied
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B5/00—Electrically-operated educational appliances
- G09B5/06—Electrically-operated educational appliances with both visual and audible presentation of the material to be studied
- G09B5/065—Combinations of audio and video presentations, e.g. videotapes, videodiscs, television systems
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Physics & Mathematics (AREA)
- Educational Administration (AREA)
- Educational Technology (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
本申请公开了一种多媒体播放方法、装置、存储介质以及电子设备，该多媒体播放方法应用于电子设备，该电子设备包括第一图像采集模组和第二图像采集模组，该方法包括：在预览界面生成第一预览框和第二预览框；将该第一图像采集模组采集的第一图像显示在该第一预览框中；在该第一图像的显示过程中，根据该第一预览框和第二图像采集模组在该第二预览框中显示第二图像；根据该第一图像和第二图像确定目标多媒体文件进行播放，从而无需依赖特定的点读笔和教材即可有效实现个别汉字或单词的单独点读，方法简单，灵活性高，用户体验性好。
The present application discloses a multimedia playback method, device, storage medium and electronic equipment. The multimedia playback method is applied to the electronic equipment. The electronic equipment includes a first image acquisition module and a second image acquisition module. The method includes: The preview interface generates a first preview frame and a second preview frame; the first image collected by the first image acquisition module is displayed in the first preview frame; during the display process of the first image, according to the first preview frame and the second image acquisition module display the second image in the second preview frame; determine the target multimedia file to play according to the first image and the second image, so that it can be effectively realized without relying on specific reading pens and teaching materials Individual reading of individual Chinese characters or words, the method is simple, the flexibility is high, and the user experience is good.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a multimedia playing method and apparatus, a storage medium, and an electronic device.
Background
The point-reading device is a reading and learning tool developed by applying a photoelectric recognition technology and combining an advanced digital voice technology.
The existing point-reading equipment is usually a camera type point-reading machine, which comprises a sound-reading device, a signal-emitting pen, a matched textbook and a camera device arranged opposite to the textbook, wherein the textbook needs to be processed by a special process, page marks are arranged on pages of the textbook, and the content of each page is provided with a longitude and latitude position. When a user clicks the page content of the textbook through the signal transmitting pen, the signal transmitting pen transmits a starting signal to the camera device to trigger the camera device to acquire images of the textbook, the acquired images are directly provided for the sound reading device to be processed, the page identification and the specific longitude and latitude coordinates of the clicked page are analyzed, corresponding voice data are called according to the page identification and the longitude and latitude coordinates to be played, and therefore the click-to-read function of the textbook is achieved, and reading is vivid and interesting.
However, the realization of the point reading function needs to depend on a specific point reading pen and a point reading teaching material, so that the cost is high and the limitation is large.
Disclosure of Invention
The embodiment of the application provides a multimedia playing method, a multimedia playing device, a storage medium and electronic equipment, which can realize a point reading function with low cost and have high flexibility.
The embodiment of the application provides a multimedia playing method, which is applied to electronic equipment, wherein the electronic equipment comprises a first image acquisition module and a second image acquisition module, and the multimedia playing method comprises the following steps:
generating a first preview frame and a second preview frame on a preview interface;
displaying a first image acquired by the first image acquisition module in the first preview frame;
in the display process of the first image, displaying a second image in the second preview frame according to the first preview frame and a second image acquisition module;
and determining a target multimedia file to play according to the first image and the second image.
The embodiment of the present application further provides a multimedia playing device, which is applied to an electronic device, the electronic device includes a first image capturing module and a second image capturing module, and the multimedia playing device includes:
the generation module is used for generating a first preview frame and a second preview frame on a preview interface;
the first display module is used for displaying a first image acquired by the first image acquisition module in the first preview frame;
the second display module is used for displaying a second image in the second preview frame according to the first preview frame and the second image acquisition module in the display process of the first image;
and the playing module is used for determining a target multimedia file to play according to the first image and the second image.
The embodiment of the application also provides a storage medium, wherein a plurality of instructions are stored in the storage medium, and the instructions are suitable for being loaded by a processor to execute any one of the multimedia playing methods.
An embodiment of the present application further provides an electronic device, which includes a processor and a memory, where the processor is electrically connected to the memory, the memory is used to store instructions and data, and the processor is used in any of the steps of the multimedia playing method.
The application provides a multimedia playing method, a multimedia playing device, a storage medium and electronic equipment, which are applied to the electronic equipment, wherein the electronic equipment comprises a first image acquisition module and a second image acquisition module, a first preview frame and a second preview frame are generated on a preview interface, a first image acquired by the first image acquisition module is displayed in the first preview frame, and in the display process of the first image, a second image is displayed in the second preview frame according to the first preview frame and the second image acquisition module, and a target multimedia file is determined according to the first image and the second image for playing, so that individual point reading of individual Chinese characters or words in a complete sentence can be effectively realized without depending on a specific point reading pen and teaching materials, the cost is low, the method is simple, the flexibility is high, and the user experience is good.
Drawings
The technical solution and other advantages of the present application will become apparent from the detailed description of the embodiments of the present application with reference to the accompanying drawings.
Fig. 1 is a scene schematic diagram of a point-reading process of an electronic device according to an embodiment of the present application.
Fig. 2 is a schematic flowchart of a multimedia playing method according to an embodiment of the present application.
Fig. 3 is another schematic flow chart of a multimedia playing method according to an embodiment of the present application.
Fig. 4 is a schematic view of a preview interface of an electronic device according to an embodiment of the present application.
Fig. 5 is a schematic flow chart illustrating implementation of a point-to-read function according to an embodiment of the present application.
Fig. 6 is a schematic diagram of an interception process of a target audio file according to an embodiment of the present application.
Fig. 7 is a schematic structural diagram of a multimedia playing device according to an embodiment of the present application.
Fig. 8 is a schematic structural diagram of a second display module according to an embodiment of the present application.
Fig. 9 is a schematic structural diagram of a play module according to an embodiment of the present application
Fig. 10 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The embodiment of the application provides a multimedia playing method, a multimedia playing device, a storage medium and electronic equipment.
A multimedia playing method is applied to electronic equipment, the electronic equipment comprises a first image acquisition module and a second image acquisition module, and the multimedia playing method comprises the following steps: generating a first preview frame and a second preview frame on a preview interface; displaying a first image acquired by the first image acquisition module in the first preview frame; in the display process of the first image, displaying a second image in the second preview frame according to the first preview frame and a second image acquisition module; and determining a target multimedia file to play according to the first image and the second image.
Wherein, the first image acquisition module and the second image acquisition module can be cameras. The multimedia file can be an audio file or a video file, which can be recorded in advance by a manufacturer according to the contents of the click-to-read textbook, and in the recording process, the manufacturer can take a picture of each recorded textbook content and store the picture and the corresponding recorded multimedia file in an associated manner, so that a subsequent user can click-to-read all or part of the contents of the multimedia file by using electronic equipment with double cameras.
Specifically, when a user needs to click and read a part of content of a multimedia file, such as a single character in a sentence, the user may use An electronic device with two cameras to shoot the multimedia file, where one camera is a long-range camera for shooting the whole sentence, and the other camera is a short-range camera for shooting the single character, and the implementation process may refer to fig. 1, for example, when the user needs to read a point, a preview interface may be entered, and a first preview box and a second preview box may be generated on the preview interface, where the first preview box is square, the second preview box is oval, at this time, the user may align the cameras with the sentence to be shot, such as n characters a1-An, so that the long-range camera displays the whole sentence in the first preview box, and then the user may click the single character (such as Ai) in the display screen, and then the electronic equipment can select a target multimedia file corresponding to the sentence from the recorded multimedia files according to the image in the first preview frame, and intercept the content corresponding to the character from the target multimedia file for playing according to the images in the first preview frame and the second preview frame, thereby realizing the click-to-read function of a single character.
As shown in fig. 2, fig. 2 is a schematic flowchart of a multimedia playing method provided in the embodiment of the present application. The multimedia playing method is applied to electronic equipment, the electronic equipment comprises a first image acquisition module and a second image acquisition module, and the specific flow can be as follows:
101. and generating a first preview frame and a second preview frame on the preview interface.
In this embodiment, the first image capturing module and the second image capturing module mainly refer to image capturing devices such as a camera. The preview interface is mainly used for displaying a shooting picture, and can be a point-reading interface of a point-reading APP. The positions of the first preview frame and the second preview frame in the preview interface may be fixed, for example, the first preview frame is located in a middle area of the preview interface, the second preview frame is located below the first preview frame, and the like. In the actual operation process, a user can enter the preview interface by clicking a designated key (such as a click-to-read key) of the click-to-read APP, and when entering the preview interface, the electronic device can be triggered to generate the first preview frame and the second preview frame.
102. And displaying the first image acquired by the first image acquisition module in the first preview frame.
In this embodiment, the first preview frame is used to display a picture taken by the first image capture module. Specifically, when a user needs to read a sentence of text at a point, the sentence of text can be shot through the first image acquisition module, and the shot content is displayed in the first preview frame so that the user can confirm the shot content.
103. And in the display process of the first image, displaying a second image in the second preview frame according to the first preview frame and the second image acquisition module.
In this embodiment, the second preview frame is used to display a picture taken by the second image capture module, the second image capture module mainly takes a close-range shot of a partial image of the first preview frame, that is, the first image is a shot of the entire content, and the second image is an enlarged image of the partial content. The shape of the first preview frame and the second preview frame may be set manually, for example, the first preview frame may be a square frame shape, the second preview frame may be a square frame shape or a circular frame shape, but may also be other shapes.
For example, the step 102 may specifically include:
acquiring click information of a user on the first preview frame;
generating a second image according to the click information, the first image and a second image acquisition module;
displaying the second image in the second preview frame.
In this embodiment, in the display process of the first image, if the user does not need to click and read the whole sentence of characters, only a single character needs to be clicked, the single character needing to be clicked and read can be found in the first preview frame and selected, and the selection mode is mainly clicking, such as clicking or double clicking.
For example, the click information includes a click position, and at this time, the step of "generating the second image according to the click information, the first image, and the second image capturing module" may specifically include:
converting the click position into an image position;
determining an enlargement area in the first image according to the image position;
and amplifying the content in the amplification area by using the second image acquisition module to obtain a second image.
In this embodiment, an enlarged region may be generated with the image position as a center, where the shape of the enlarged region may be a circle, a square, or another shape, and the size of the enlarged region may be set in advance by a human, or may be determined according to the actual font size in the first preview frame, for example, the larger the area occupied by a single font in the first preview frame is, the larger the enlarged region is, and generally, the area of the enlarged region is optimal to accommodate the single font. The second image acquisition module can be digitally zoomed to realize the amplification function, namely, the focal length of the second image acquisition module is enlarged, which is similar to the effect achieved when a user double clicks a picture.
104. And determining a target multimedia file to play according to the first image and the second image.
In this embodiment, the user may be informed of the currently captured target content through the images displayed in the first preview frame and the second preview frame, so that the user may confirm whether the adjustment is needed, and if not, the user may end the image capturing by clicking a key (such as a confirmation key) on the preview interface, so that the electronic device may obtain the corresponding click-to-read content according to the finally captured first image and the second image.
For example, when the shooting is finished, the step 104 may specifically include:
(1) and determining a target multimedia file according to the first image.
For example, the step (1) may specifically include:
matching the first image with a preset image library, and acquiring a successfully matched preset image;
and searching a corresponding multimedia file according to the successfully matched preset image to serve as a target multimedia file.
In this embodiment, the multimedia file may be an audio file or a video file. The preset image in the preset image library can be set by a manufacturer in advance according to a click-to-read textbook, for example, the manufacturer can photograph each sentence, paragraph or article in the click-to-read textbook in advance, and store the photograph as the preset image in the preset image library, meanwhile, the manufacturer can record audio or video for each sentence, paragraph or article, and then store the recorded audio or video as a multimedia file and a corresponding photograph in an associated manner, so that the corresponding multimedia file can be found subsequently according to the photograph, and click-to-read of the multimedia file can be realized through photographing.
(2) And extracting the point-reading content from the target multimedia file according to the second image and the first image.
For example, the step (2) may specifically include:
performing character recognition on the first image and the second image;
matching the characters identified in the second image with the characters identified in the first image;
determining the arrangement positions of the successfully matched characters in all the characters in the first image;
and extracting the point reading content from the target multimedia file according to the arrangement position.
In this embodiment, the characters may include chinese characters and words, and the arrangement position may be represented in a proportional form, such as 7:14, which represents that 14 characters are included in the first image, and the characters are arranged at the 7 th position. The character recognition mainly comprises character area determination and character segmentation, and because the second image is an enlarged image of a certain area in the first image, characters in the second image are necessarily contained in the first image. The character matching mainly comprises similarity matching, the similarity of characters in the first image and characters in the second image needs to be calculated in sequence according to the arrangement sequence of the characters, and the characters with high similarity are used as the characters which are successfully matched.
For example, the step of "extracting the read-and-click content from the target multimedia file according to the arrangement position" may specifically include:
acquiring the total playing time of the target multimedia file;
determining an interception time period from the total playing time length according to the arrangement position;
and extracting the multimedia content corresponding to the interception time period from the multimedia file to be used as point reading content.
In this embodiment, the multimedia file mainly refers to a voice file, and since the speed of speech and the interval duration of each character are fixed when the speech reader records and reads the textbook aloud, the whole speech is easily segmented according to the total playing duration and the total word number to obtain the aloud speech of each character.
(3) And playing the click-to-read content.
In this embodiment, the position of the click-to-read word in the whole click-to-read sentence is determined by the first image acquisition module and the second image acquisition module, and a required part is intercepted from the recorded whole voice based on the position, so that voice recording is not required for each word, and the method is simple.
From the above, the multimedia playing method provided by this embodiment is applied to an electronic device, the electronic device includes a first image capturing module and a second image capturing module, a first preview frame and a second preview frame are generated on a preview interface, a first image captured by the first image capturing module is displayed in the first preview frame, and during the displaying of the first image, a second image is displayed in the second preview frame according to the first preview frame and the second image capturing module, and a target multimedia file is determined according to the first image and the second image for playing, so that a required portion can be extracted from a recorded whole multimedia file according to a click-to-read requirement of a user, and individual click-to-read of individual characters or words in a complete sentence is effectively achieved, the method is simple, high in flexibility, good in user experience, and does not need to rely on a specific click-to-read pen and teaching materials, the cost is low.
In this embodiment, the multimedia playing apparatus will be specifically integrated in an electronic device as an example for detailed description.
Referring to fig. 3, a multimedia playing method is applied to an electronic device, where the electronic device includes a first image capturing module and a second image capturing module, and a specific process may be as follows:
201. the electronic equipment generates a first preview frame and a second preview frame on a preview interface.
For example, the first image capturing module and the second image capturing module are cameras, and the display positions of the first preview frame and the second preview frame on the preview interface may be fixed, for example, referring to fig. 4, the first preview frame may be located approximately at the upper position in the middle of the preview interface, and the second preview frame is located at the lower right of the first preview frame. The first preview box and the second preview box may be sized according to the size of the preview interface, for example, in fig. 3, the first preview box may be one third of the size of the preview interface, the second preview box may be one tenth of the size of the preview interface, and so on.
202. The electronic equipment displays the first image collected by the first image collecting module in the first preview frame.
For example, referring to fig. 5, the user may shoot the first image capturing module aiming at the content a in the click-to-read textbook, so that the content a may be displayed in the first preview frame, where the content a may be a piece of text or a sentence, which may include n pieces of text a 1-An.
203. In the display process of the first image, the electronic equipment acquires a click position of a user for clicking the first preview frame, and converts the click position into an image position.
For example, during the display of the content a, if the user only wants to click and read for individual characters in the content a, such as Ai, i ∈ (1, n), the user can click on the character Ai in the first preview frame.
204. The electronic equipment determines an amplification area in the first image according to the image position, performs amplification processing on the content in the amplification area by using the second image acquisition module to obtain a second image, and then displays the second image in the second preview frame.
For example, the shape of the second preview pane may correspond to the shape of the magnified region. The electronic equipment can draw a circle by taking the position of Ai in the first image as the center, the radius of the circle is slightly larger than the font size of Ai, the obtained area is an amplification area, then, the focal length of the second image acquisition module is adjusted to a specified value, the position of the amplification area in the textbook is shot by utilizing the adjusted second image acquisition module, and the image is displayed in the second preview frame while shooting so as to be convenient for a user to confirm.
205. The electronic equipment matches the first image with a preset image library, acquires a successfully matched preset image, and then searches a corresponding multimedia file according to the successfully matched preset image to serve as a target multimedia file.
For example, the multimedia files are mainly audio files. When the content displayed in the second image is the text Ai that the user needs to click and read, the user can click a confirmation key (or a photographing key) on the preview interface to end the photographing, and at this time, the electronic device can find the audio file corresponding to the n texts a1-An according to the first image when the photographing is ended. Of course, before the photographing is finished, the images in the first preview frame and the second preview frame are changed continuously following the adjustment of the user, and the background usually acquires only the image at the last moment when the photographing is finished as the first image and the second image.
206. The electronic equipment identifies characters of the first image and the second image, and matches the characters identified in the second image with the characters identified in the first image.
207. The electronic equipment determines the arrangement positions of the successfully matched characters in all the characters in the first image, and acquires the total playing time of the target multimedia file.
For example, it can be known that n characters A1-An are included in the first image, and a character Ai is included in the second image, and the arrangement position is i/n, by character recognition and matching.
208. The electronic equipment determines an interception time period from the total playing time period according to the arrangement position, extracts multimedia content corresponding to the interception time period from the multimedia file to be used as click-to-read content, and then plays the click-to-read content.
For example, referring to fig. 6, if the total playing time is T, the clipping time period may be [ (i-1)/n ] T to [ (i +1)/n ] T, and the audio content clipped according to the clipping time period is the click-to-read audio of the text Ai.
As can be seen from the above, the multimedia playing method provided in this embodiment is applied to an electronic device, where the electronic device includes a first image capturing module and a second image capturing module, where the electronic device can generate a first preview frame and a second preview frame on a preview interface, and display a first image captured by the first image capturing module in the first preview frame, then, in a display process of the first image, obtain a click position where a user clicks the first preview frame, convert the click position into an image position, then, determine an enlargement area in the first image according to the image position, and enlarge content in the enlargement area by using the second image capturing module to obtain a second image, then display the second image in the second preview frame, then, match the first image with a preset image library, acquiring a preset image which is successfully matched, searching a corresponding multimedia file according to the preset image which is successfully matched to be used as a target multimedia file, then carrying out character recognition on the first image and the second image, matching characters recognized in the second image with characters recognized in the first image, then determining the arrangement position of the successfully matched characters in all the characters in the first image, acquiring the total playing time of the target multimedia file, then determining an interception time period from the total playing time period according to the arrangement position, extracting multimedia content corresponding to the interception time period from the multimedia file to be used as click-to-read content, and then playing the click-to-read content, so that a required part can be extracted from the recorded whole multimedia file according to the click-to-read requirements of a user, and effectively realizing the individual click-to-read of individual Chinese characters or words in a complete sentence, the method is simple, high in flexibility, good in user experience, low in cost and free of dependence on a specific touch and talk pen and teaching materials.
According to the method described in the foregoing embodiment, the present embodiment will be further described from the perspective of a multimedia playing apparatus, which may be specifically implemented as a stand-alone entity, or may be implemented by being integrated in an electronic device, such as a terminal, where the terminal may include a mobile phone, a tablet computer, a personal computer, and the like.
Referring to fig. 7, fig. 7 specifically illustrates a multimedia playing device provided in an embodiment of the present application, which is applied to an electronic device, where the electronic device includes a first image capturing module and a second image capturing module, and the multimedia playing device may include: the display device comprises a generation module 10, a first display module 20, a second display module 30 and a playing module 40, wherein:
(1) generating module 10
The generating module 10 is configured to generate a first preview frame and a second preview frame on a preview interface.
In this embodiment, the first image capturing module and the second image capturing module mainly refer to image capturing devices such as a camera. The preview interface is mainly used for displaying a shooting picture, and can be a point-reading interface of a point-reading APP. The positions of the first preview frame and the second preview frame in the preview interface may be fixed, for example, the first preview frame is located in a middle area of the preview interface, the second preview frame is located below the first preview frame, and the like. In the actual operation process, a user may enter the preview interface by clicking a designated key (e.g., a click-to-read key) of the click-to-read APP, and when entering the preview interface, the generation module 10 may be triggered to generate the first preview frame and the second preview frame.
(2) First display module 20
The first display module 20 is configured to display the first image acquired by the first image acquisition module in the first preview frame.
In this embodiment, the first preview frame is used to display a picture taken by the first image capture module. Specifically, when the user needs to read a sentence of text, the first display module 20 may capture the sentence of text through the first image capturing module, and display the captured content in the first preview frame, so that the user can confirm the captured content.
(3) Second display module 30
And the second display module 30 is configured to display a second image in the second preview frame according to the first preview frame and the second image capture module in the display process of the first image.
In this embodiment, the second preview frame is used to display a picture taken by the second image capture module, the second image capture module mainly takes a close-range shot of a partial image of the first preview frame, that is, the first image is a shot of the entire content, and the second image is an enlarged image of the partial content. The shape of the first preview frame and the second preview frame may be set manually, for example, the first preview frame may be a square frame shape, the second preview frame may be a square frame shape or a circular frame shape, but may also be other shapes.
For example, referring to fig. 8, the second display module 30 may include an obtaining sub-module 31, a generating sub-module 32, and a displaying sub-module 33, wherein:
the obtaining submodule 31 is configured to obtain click information of the first preview box from the user;
the generation submodule 32 is used for generating a second image according to the click information, the first image and the second image acquisition module;
and a display sub-module 33 for displaying the second image in the second preview frame.
In this embodiment, in the display process of the first image, if the user does not need to click and read the whole sentence of characters, only a single character needs to be clicked, the single character needing to be clicked and read can be found in the first preview frame and selected, and the selection mode is mainly clicking, such as clicking or double clicking.
For example, the click information includes a click position, and the generating submodule 32 may specifically be configured to:
converting the click position into an image position;
determining an enlargement area in the first image according to the image position;
and amplifying the content in the amplification area by using the second image acquisition module to obtain a second image.
In this embodiment, the generating sub-module 32 may generate a magnified area with the image position as the center, where the magnified area may be in a shape of a circle, a square, or another shape, and the size of the magnified area may be set in advance by a human, or may be determined according to the actual font size in the first preview frame, for example, the larger the area occupied by a single font in the first preview frame is, the larger the magnified area is, and generally, the area of the magnified area is optimal to accommodate the single font. The second image acquisition module can be digitally zoomed to realize the amplification function, namely, the focal length of the second image acquisition module is enlarged, which is similar to the effect achieved when a user double clicks a picture.
(4) Play module 40
And the playing module 40 is used for determining the target multimedia file to play according to the first image and the second image.
In this embodiment, the user may be informed of the currently captured target content through the images displayed in the first preview frame and the second preview frame, so that the user may confirm whether the adjustment is needed, and if not, the user may end the image capturing by clicking a key (such as a confirmation key) on the preview interface, so that the electronic device may obtain the corresponding click-to-read content according to the finally captured first image and the second image.
For example, when the shooting is finished, please refer to fig. 9, the playing module 40 may specifically include a determining sub-module 41, an extracting sub-module 42, and a playing sub-module 43, where:
a determining sub-module 41 for determining the target multimedia file from the first image.
For example, the determination submodule 41 may be specifically configured to:
matching the first image with a preset image library, and acquiring a successfully matched preset image;
and searching a corresponding multimedia file according to the successfully matched preset image to serve as a target multimedia file.
In this embodiment, the multimedia file may be an audio file or a video file. The preset image in the preset image library may be set by the manufacturer in advance according to the click-to-read textbook, for example, the manufacturer may take a picture of each sentence, paragraph, or article in the click-to-read textbook in advance, and store the picture as the preset image in the preset image library, meanwhile, the manufacturer may record an audio or video for each sentence, paragraph, or article, and then store the recorded audio or video as the multimedia file and the corresponding picture in association, so that the subsequent determining sub-module 41 may find the corresponding multimedia file according to the picture, that is, may take a picture to realize the click-to-read of the multimedia file.
And the extraction submodule 42 is used for extracting the reading contents from the target multimedia file according to the second image and the first image.
For example, the extraction sub-module 42 may be specifically configured to:
performing character recognition on the first image and the second image;
matching the characters identified in the second image with the characters identified in the first image;
determining the arrangement positions of the successfully matched characters in all the characters in the first image;
and extracting the point reading content from the target multimedia file according to the arrangement position.
In this embodiment, the characters may include chinese characters and words, and the arrangement position may be represented in a proportional form, such as 7:14, which represents that 14 characters are included in the first image, and the characters are arranged at the 7 th position. The character recognition mainly comprises character area determination and character segmentation, and because the second image is an enlarged image of a certain area in the first image, characters in the second image are necessarily contained in the first image. The character matching mainly comprises similarity matching, the similarity of characters in the first image and characters in the second image needs to be calculated in sequence according to the arrangement sequence of the characters, and the characters with high similarity are used as the characters which are successfully matched.
For example, the extraction sub-module may be further operable to:
acquiring the total playing time of the target multimedia file;
determining an interception time period from the total playing time length according to the arrangement position;
and extracting the multimedia content corresponding to the interception time period from the multimedia file to be used as point reading content.
In this embodiment, the multimedia file mainly refers to a voice file, and since the speed of speech and the interval duration of each character are fixed when the speech reader records and reads the textbook aloud, the whole speech is easily segmented according to the total playing duration and the total word number to obtain the aloud speech of each character.
And a playing sub-module 43 for playing the read-on content.
In this embodiment, the position of the click-to-read word in the whole click-to-read sentence is determined by the first image acquisition module and the second image acquisition module, and a required part is intercepted from the recorded whole voice based on the position, so that voice recording is not required for each word, and the method is simple.
In a specific implementation, the above units may be implemented as independent entities, or may be combined arbitrarily to be implemented as the same or several entities, and the specific implementation of the above units may refer to the foregoing method embodiments, which are not described herein again.
As can be seen from the above, the multimedia playing apparatus provided in this embodiment is applied to an electronic device, the electronic device includes a first image capturing module and a second image capturing module, a first preview frame and a second preview frame are generated on a preview interface through a generating module 10, a first display module 20 displays a first image captured by the first image capturing module in the first preview frame, and during the displaying process of the first image, a second image is displayed in the second preview frame through a second display module 30 according to the first preview frame and the second image capturing module, a playing module 40 determines a target multimedia file to play according to the first image and the second image, so as to extract a required portion from a recorded whole multimedia file according to a click-to-read requirement of a user, thereby effectively achieving individual click-to-read of an individual chinese character or word in a complete sentence, the method is simple, high in flexibility, good in user experience, low in cost and free of dependence on a specific touch and talk pen and teaching materials.
In addition, the embodiment of the application also provides electronic equipment which can be equipment such as a smart phone and a tablet computer. As shown in fig. 10, the electronic device 900 includes a processor 901, a memory 902, a display 903, and a control circuit 904. The processor 901 is electrically connected to the memory 902, the display 903, and the control circuit 904.
The processor 901 is a control center of the electronic device 900, connects various parts of the whole electronic device by using various interfaces and lines, executes various functions of the electronic device and processes data by running or loading an application program stored in the memory 902 and calling the data stored in the memory 902, thereby performing overall monitoring of the electronic device.
In this embodiment, the processor 901 in the electronic device 900 loads instructions corresponding to processes of one or more application programs into the memory 902 according to the following steps, and the processor 901 runs the application programs stored in the memory 902, so as to implement various functions:
generating a first preview frame and a second preview frame on a preview interface;
displaying a first image acquired by the first image acquisition module in the first preview frame;
in the display process of the first image, displaying a second image in the second preview frame according to the first preview frame and a second image acquisition module;
and determining a target multimedia file to play according to the first image and the second image.
The display 903 may be used to display information input by or provided to the user as well as various graphical user interfaces of the terminal, which may be comprised of images, text, icons, video, and any combination thereof.
The control circuit 904 is electrically connected to the display 903, and is configured to control the display 903 to display information.
In some embodiments, as shown in fig. 10, the electronic device 900 further comprises: a radio frequency circuit 905, an input unit 906, an audio circuit 907, a sensor 908, and a power supply 909. The processor 901 is electrically connected to the rf circuit 905, the input unit 906, the audio circuit 907, the sensor 908, and the power source 909.
The radio frequency circuit 905 is configured to receive and transmit radio frequency signals, so as to establish wireless communication with a network device or other electronic devices through wireless communication, and receive and transmit signals with the network device or other electronic devices.
The input unit 906 may be used to receive input numbers, character information, or user characteristic information (e.g., a fingerprint), and to generate keyboard, mouse, joystick, optical, or trackball signal inputs related to user settings and function control. The input unit 906 may include a fingerprint recognition module.
The audio circuit 907 may provide an audio interface between the user and the terminal through a speaker, microphone, or the like.
The electronic device 900 may also include at least one sensor 908, such as light sensors, motion sensors, and other sensors. Specifically, the light sensor may include an ambient light sensor that may adjust the brightness of the display panel according to the brightness of ambient light, and a proximity sensor that may turn off the display panel and/or the backlight when the terminal is moved to the ear. As one of the motion sensors, the gravity acceleration sensor can detect the magnitude of acceleration in each direction (generally, three axes), can detect the magnitude and direction of gravity when the mobile phone is stationary, and can be used for applications of recognizing the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer and tapping), and the like; as for other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, and an infrared sensor, which can be configured in the terminal, detailed description is omitted here.
The power supply 909 is used to supply power to the various components of the electronic device 900. In some embodiments, the power source 909 may be logically connected to the processor 901 through a power management system, so that functions of managing charging, discharging, and power consumption management are realized through the power management system.
Although not shown in fig. 10, the electronic device 900 may further include a camera, a bluetooth module, etc., which are not described in detail herein.
It will be understood by those skilled in the art that all or part of the steps of the methods of the above embodiments may be performed by instructions or by associated hardware controlled by the instructions, which may be stored in a computer readable storage medium and loaded and executed by a processor. To this end, the present invention provides a storage medium, in which a plurality of instructions are stored, and the instructions can be loaded by a processor to execute the steps in any one of the multimedia playing methods provided by the embodiments of the present invention.
Wherein the storage medium may include: read Only Memory (ROM), Random Access Memory (RAM), magnetic or optical disks, and the like.
Since the instructions stored in the storage medium can execute the steps in any multimedia playing method provided in the embodiments of the present invention, the beneficial effects that can be achieved by any multimedia playing method provided in the embodiments of the present invention can be achieved, which are detailed in the foregoing embodiments and will not be described herein again.
The above operations can be implemented in the foregoing embodiments, and are not described in detail herein.
In summary, although the present application has been described with reference to the preferred embodiments, the above-described preferred embodiments are not intended to limit the present application, and those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present application, so that the scope of the present application shall be determined by the appended claims.
Claims (12)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711085728.9A CN107731020B (en) | 2017-11-07 | 2017-11-07 | Multimedia playing method, device, storage medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711085728.9A CN107731020B (en) | 2017-11-07 | 2017-11-07 | Multimedia playing method, device, storage medium and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107731020A CN107731020A (en) | 2018-02-23 |
| CN107731020B true CN107731020B (en) | 2020-05-12 |
Family
ID=61222839
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711085728.9A Expired - Fee Related CN107731020B (en) | 2017-11-07 | 2017-11-07 | Multimedia playing method, device, storage medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107731020B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109214379B (en) * | 2018-10-23 | 2022-02-15 | 昆明微想智森科技股份有限公司 | Multifunctional point reading pointing piece based on image recognition tracking technology and point reading method |
| CN110046009B (en) * | 2019-02-19 | 2022-08-23 | 创新先进技术有限公司 | Recording method, recording device, server and readable storage medium |
| CN110059678A (en) * | 2019-04-17 | 2019-07-26 | 上海肇观电子科技有限公司 | A kind of detection method, device and computer readable storage medium |
| CN111077992B (en) * | 2019-06-09 | 2023-06-27 | 广东小天才科技有限公司 | Click-to-read method, electronic equipment and storage medium |
| CN110634342A (en) * | 2019-09-19 | 2019-12-31 | 广东小天才科技有限公司 | A method, system, storage medium and electronic device for identifying an outline frame |
| CN110767000A (en) * | 2019-10-28 | 2020-02-07 | 安徽信捷智能科技有限公司 | Children's course synchronizer based on image recognition |
| CN113949815B (en) * | 2021-11-17 | 2024-10-25 | 维沃移动通信有限公司 | Shooting preview method, device and electronic equipment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101493996A (en) * | 2009-01-15 | 2009-07-29 | 北方工业大学 | An intelligent reader and its realization method |
| CN104253904A (en) * | 2014-09-04 | 2014-12-31 | 广东小天才科技有限公司 | Method for realizing point-reading learning and smart phone |
| CN106710326A (en) * | 2015-07-31 | 2017-05-24 | 曾晓敏 | Reading textbook system based on Chinese |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130314558A1 (en) * | 2012-05-24 | 2013-11-28 | Mediatek Inc. | Image capture device for starting specific action in advance when determining that specific action is about to be triggered and related image capture method thereof |
| US20140127667A1 (en) * | 2012-11-05 | 2014-05-08 | Marco Iannacone | Learning system |
-
2017
- 2017-11-07 CN CN201711085728.9A patent/CN107731020B/en not_active Expired - Fee Related
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101493996A (en) * | 2009-01-15 | 2009-07-29 | 北方工业大学 | An intelligent reader and its realization method |
| CN104253904A (en) * | 2014-09-04 | 2014-12-31 | 广东小天才科技有限公司 | Method for realizing point-reading learning and smart phone |
| CN106710326A (en) * | 2015-07-31 | 2017-05-24 | 曾晓敏 | Reading textbook system based on Chinese |
Non-Patent Citations (1)
| Title |
|---|
| 文本定位算法及其在智能读书软件中的应用;吴治新;《中国优秀硕士学位论文全文数据库》;20170315;23-48 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107731020A (en) | 2018-02-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107731020B (en) | Multimedia playing method, device, storage medium and electronic equipment | |
| CN107835366B (en) | Multimedia playback method, device, storage medium and electronic device | |
| CN110740259B (en) | Video processing method and electronic device | |
| CN110381371B (en) | A video editing method and electronic device | |
| CN111541845B (en) | Image processing method and device and electronic equipment | |
| CN111182205B (en) | Shooting method, electronic device and medium | |
| JP2021524957A (en) | Image processing methods and their devices, terminals and computer programs | |
| WO2018141144A1 (en) | Method for use in processing text and voice information, and terminal | |
| CN107864353B (en) | A video recording method and mobile terminal | |
| CN107809598B (en) | A kind of image pickup method, mobile terminal and server | |
| WO2012011614A1 (en) | Information device, control method thereof and system | |
| CN111010510A (en) | A shooting control method, device and electronic device | |
| CN111445927B (en) | Audio processing method and electronic equipment | |
| CN110855893A (en) | A method and electronic device for shooting video | |
| CN111182211B (en) | Shooting method, image processing method, and electronic device | |
| CN107871000B (en) | Audio playback method, device, storage medium and electronic device | |
| CN107885482B (en) | Audio playback method, device, storage medium and electronic device | |
| CN108984143B (en) | Display control method and terminal equipment | |
| CN109257649B (en) | A kind of multimedia file generation method and terminal device | |
| CN107948729A (en) | Rich media processing method, device, storage medium and electronic equipment | |
| CN111783175A (en) | Display interface privacy protection method, terminal and computer-readable storage medium | |
| CN109460556A (en) | A kind of interpretation method and device | |
| CN108287895B (en) | A method and system for searching questions based on explicit frame adjustment, and a handheld photographing device | |
| CN113873165A (en) | Photographing method and device and electronic equipment | |
| CN110019897B (en) | Method and device for displaying picture |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: Changan town in Guangdong province Dongguan 523860 usha Beach Road No. 18 Applicant after: GUANGDONG OPPO MOBILE TELECOMMUNICATIONS Corp.,Ltd. Address before: Changan town in Guangdong province Dongguan 523860 usha Beach Road No. 18 Applicant before: GUANGDONG OPPO MOBILE TELECOMMUNICATIONS Corp.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20200512 |