CN111918145B - Video segmentation method and video segmentation device - Google Patents

Abstract

The application provides a video segmentation method and a video segmentation device, wherein the method comprises the following steps: the video segmenting device segments the video to be processed according to at least one of content description information used for describing the content of the video to be processed and a presentation demonstrated in the video to be processed uploaded in advance and voice information of the video to be processed. The technical scheme can be combined with information except the content of the video to be processed to segment the video to be processed, so that the accuracy of segmentation can be improved.

Description

Video segmentation method and video segmentation device

technical field

The present application relates to the field of information technology, and more particularly, to a video segmentation method and a video segmentation device.

Background technique

In order to conveniently watch the video, a complete video can be divided into multiple segments. In this way, the user can directly view the segment of interest.

A common video segmentation method at present is to segment the video based on the text information in the video. The text information in the video may be subtitles in the video, or text obtained by performing speech recognition on the video. In other words, the current basis for segmenting a video comes from the video itself. In addition, the current video segmentation based on the text information in the video needs to acquire all the text information of the video. The video stream of live video is generated in real time. Therefore, only after the end of the live video, the full text information of the video can be obtained. Therefore, the above method cannot perform real-time segmentation of live video. In addition, the above method only segments the video according to the text information of the video. This may result in that the determined segmentation point is not necessarily a suitable segmentation point.

SUMMARY OF THE INVENTION

The present application provides a video segmentation method and a video segmentation device, which can improve the accuracy of video segmentation.

In a first aspect, an embodiment of the present application provides a video segmentation method, including: a video segmentation device obtains text information of a video to be processed and voice information of the video to be processed, wherein the text information includes a presentation in the video to be processed at least one of the content description information of the manuscript and the video to be processed; the video segmentation device determines the segmentation point of the video to be processed according to the text information and the voice information; the video segmentation device, according to the segmentation point, Segment the video to be processed. The above technical solution can combine information other than the content of the video to be processed to segment the to-be-processed video, thereby improving the accuracy of the segmentation.

With reference to the first aspect, in a possible implementation manner of the first aspect, in the case that the text information includes the presentation, the video segmentation device determines, according to the text information and the voice information, the The segmentation point includes: determining a switching point of the presentation, where the presentation presents different contents before and after the switching point; determining at least one pause point according to the voice information; determining according to the switching point and the at least one pause point the segment point. Switching presentations often means that the content of the speaker's speech has changed. Therefore, the above technical solution divides the video to be processed into different segments by considering the changes of the presentation, so that the segment points of the video to be processed can be reasonably and quickly determined. In addition, when determining the segment point of the video to be processed in the above technical solution, it only needs to be based on the switching point of the presentation and the pause point near the switching point. Therefore, in the above technical solution, the video can be segmented without acquiring the completed video file. In other words, the video to be processed can be segmented in real time by using the above technical solution. Therefore, the above technical solution can be applied to the segmentation processing of live video.

With reference to the first aspect, in a possible implementation manner of the first aspect, the determining the segment point according to the switching point and the at least one pause point includes: in determining the switching point and the at least one pause point In the case of the same pause point, the switching point is determined as the segmentation point; in the case of determining that any one of the at least one pause point is different from the switch point, the at least one pause point is determined. The pause point closest to the switching point is the segment point.

With reference to the first aspect, in a possible implementation manner of the first aspect, the determining of the switching point of the presentation includes: determining the moment when a switching signal for instructing to switch the content of the presentation is obtained as the switching point .

With reference to the first aspect, in a possible implementation manner of the first aspect, the text information further includes the content description information, and the video segmentation device determines the segmentation of the to-be-processed video according to the text information and the voice information Before the paragraph point, the method further includes: determining that the presentation duration of the current page of the presentation is less than or equal to the first preset duration and greater than the second preset duration.

With reference to the first aspect, in a possible implementation manner of the first aspect, when the text information includes the content description information, the video segmentation device determines the to-be-processed video according to the text information and the voice information The segmentation point includes: determining the segmentation point of the to-be-processed video according to the voice information, the keywords of the content description information, and the pause point in the voice information. The content description information is information input in advance by the user to describe the video to be processed. The content description information may generally include some key information in the video to be processed, such as keywords, key content, and the like. Therefore, the key content described in different segments of the video to be processed can be more accurately determined based on the content description information, so that the video to be processed can be segmented more accurately.

With reference to the first aspect, in a possible implementation manner of the first aspect, the voice information includes a first voice information fragment and a second voice information fragment, wherein the second voice information fragment is before the first voice information fragment And for the voice information fragment adjacent to the first voice information fragment, according to the voice information, the keyword of the content description information and the pause point in the voice information, determine the segmentation point of the video to be processed, including: according to the The first segment of voice information, the second segment of voice information, the keyword of the content description information, and the pause point in the voice information, determine the first segment point, wherein the segment point of the video to be processed includes the first segment point. paragraph point. In addition, when determining the segmentation point of the video to be processed in the above technical solution, the position of the segmentation point can be determined only based on the keywords of the content description information and the voice information in the two adjacent video segments. The division of video segments can be implemented according to fixed time and step size. Therefore, during the video playback process, video segments can be divided for the played video. In this way, the video can be segmented without acquiring the completed video file. In other words, the video to be processed can be segmented in real time by using the above technical solution. Therefore, the above technical solution can be applied to the segmentation processing of live video.

With reference to the first aspect, in a possible implementation manner of the first aspect, according to the first voice information fragment, the second voice information fragment, the keywords of the content description information, and the pause point in the voice information, determine The first segmentation point includes: according to the keywords of the first voice information segment, the keywords of the second voice information segment, the content of the first voice information segment, the content of the second voice information segment, and the content description information keywords, determine the similarity between the first voice information segment and the second voice information segment; determine that the similarity between the first voice information segment and the second voice information segment is less than the similarity threshold; The pause point is determined to determine the first segment point.

With reference to the first aspect, in a possible implementation manner of the first aspect, the pause point in the voice information includes a pause point in the first voice information segment or a pause point adjacent to the first voice information segment, Determining the first segment point according to the pause points in the voice information includes: according to the number of pause points in the first voice information segment, the number of pause points adjacent to the first voice information segment, the pause duration, and the At least one of the words adjacent to the pause point determines the first segmentation point.

With reference to the first aspect, in a possible implementation manner of the first aspect, the pause points in the first voice information segment include K, or, the pause points adjacent to the first voice information segment include K. determining the first segment point according to at least one of the number of pause points in the first voice information segment, the number of pause points adjacent to the first speech information segment, the pause duration and the words adjacent to the pause point Including: when K is equal to 1, determining the K pause points as the segment points; including a preset in the K words where K is a positive integer greater than or equal to 2 and adjacent to the K pause points In the case of a word, determine that the pause point adjacent to the one preset word is the segment point; in the case where K is a positive integer greater than or equal to 2 and the K words include at least two of the preset words , determine the pause point with the longest pause time among the at least two pause points adjacent to at least two of the preset words as the segment point; where K is a positive integer greater than or equal to 2 and the K words do not include In the case of the preset word, it is determined that the pause point with the longest pause duration among the K pause points is the segment point.

With reference to the first aspect, in a possible implementation manner of the first aspect, the text information further includes the presentation, and the video segmentation device determines the segment of the to-be-processed video according to the text information and the voice information Before the point, the method further includes: determining that the presentation duration of the current page of the presentation is greater than a first preset duration; or determining that the presentation duration of the current page of the presentation is less than or equal to a second preset duration. The above technical solution can avoid occurrence of inappropriate segmentation caused by the presentation being unchanged for a long time or changing very rapidly.

With reference to the first aspect, in a possible implementation manner of the first aspect, the method further includes: the video segmentation device according to the content of the segmented voice information, the keywords of the segmented voice information and the keywords of the target text , determining an abstract of the segment, wherein the target text includes at least one of the presentation and the content description information. Based on the above technical solutions, the user can use the summary to quickly determine the desired position when viewing the video. In addition, in the above technical solution, information other than the video to be processed is considered in the process of determining the digest. This improves the accuracy of the digests determined and the speed with which the digests are determined.

With reference to the first aspect, in a possible implementation manner of the first aspect, the video segmentation device determines the segment according to the content of the segmented speech information, the keywords of the segmented speech information, and the keywords of the target text The abstract includes: determining a third keyword vector according to the content of the segmented speech information, the keywords of the segmented speech information and the keywords of the target text; and determining the abstract of the segment according to the third keyword vector.

With reference to the first aspect, in a possible implementation manner of the first aspect, the video segmentation device determines the abstract of the segment according to the third keyword vector, including: according to the target text and the segmented voice information , determine the reference text, wherein the reference text includes J sentences, and J is a positive integer greater than or equal to 1; according to the keywords of the segmented speech information, the keywords of the target text and each sentence in the J sentences , determine J keyword vectors; according to the third keyword vector and the J keyword vectors, determine the abstract of the segment.

With reference to the first aspect, in a possible implementation manner of the first aspect, determining the reference text according to the target text and the segmented speech information includes: when the target text includes redundant sentences, adding The redundant sentence in the target text is deleted, and the revised target text is obtained and the revised target text is merged with the segmented speech information to obtain the reference text; when the target text does not include the redundant sentence, The target text is combined with the segmented speech information to obtain the reference text.

With reference to the first aspect, in a possible implementation manner of the first aspect, determining the abstract of the segment according to the third keyword vector and the J keyword vectors includes: according to the third keyword vector and the J keyword vectors For the J keyword vectors, determine J distances, wherein the jth distance in the J distances is determined according to the third keyword vector and the jth keyword vector in the J keyword vectors, j is a positive integer greater than or equal to 1 and less than or equal to J; determine the R distances with the shortest distances among the J distances, where R is a positive integer greater than or equal to 1 and less than J; determine the summary of the segment, where the score The summary of the segment includes sentences corresponding to the R distances.

In combination with the first aspect, in a possible implementation manner of the first aspect, the video to be processed is a real-time video stream, and the voice information of the video to be processed is the real-time video stream from the starting moment of the real-time video stream or the last time. Voice information from the segment point to the current moment. The above technical solution can realize real-time segmentation of video. In other words, when using the above technical solution to segment a video, it is not necessary to acquire the entire content of the video to be processed. Therefore, the above technical solution can realize real-time segmentation of live video.

In a second aspect, an embodiment of the present application provides a video segmentation apparatus, where the apparatus includes a unit for executing the first aspect or any possible implementation manner of the first aspect.

Optionally, the video segmentation apparatus of the second aspect may be a computer device, or may be a component (eg, a chip or a circuit, etc.) that can be used in a computer device.

In a third aspect, an embodiment of the present application provides a storage medium, where the storage medium stores an instruction for implementing the method described in the first aspect or any possible implementation manner of the first aspect.

In a fourth aspect, the embodiments of the present application provide a computer program product containing instructions, when the computer program product is run on a computer, the computer program product enables the computer to execute the first aspect or any of the possible implementations of the first aspect. method described.

Description of drawings

1 is a schematic diagram of a system to which a video segmentation method provided by an embodiment of the present application can be applied;

2 is a schematic diagram of another system to which the video segmentation method provided by the embodiment of the present application can be applied;

3 is a schematic flowchart of a video segmentation method provided according to an embodiment of the present application;

4 is a schematic diagram of a video conference process provided according to an embodiment of the present application;

5 is a schematic flowchart of a video segmentation method provided according to an embodiment of the present application;

6 is a schematic flowchart of a method for video segmentation provided according to an embodiment of the present application;

7 is a structural block diagram of a video segmentation apparatus provided according to an embodiment of the present application;

FIG. 8 is a structural block diagram of a video segmentation apparatus provided according to an embodiment of the present application.

Detailed ways

The technical solutions in the present application will be described below with reference to the accompanying drawings.

In this application, "at least one" means one or more, and "plurality" means two or more. "And/or", which describes the association relationship of the associated objects, indicates that there can be three kinds of relationships, for example, A and/or B, which can indicate: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A, B can be singular or plural. The character "/" generally indicates that the associated objects are an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of a single item(s) or a plurality of items(s). For example, at least one (a) of a, b or c may represent: a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, and c may be single or multiple. In addition, in the embodiments of the present application, words such as "first" and "second" do not limit the quantity and execution order.

It should be noted that, in this application, words such as "exemplary" or "for example" are used to represent examples, illustrations or illustrations. Any embodiment or design described in this application as "exemplary" or "such as" should not be construed as preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "such as" is intended to present the related concepts in a specific manner.

Various aspects or features of the present application may be implemented as methods, apparatus, or articles of manufacture using standard programming and/or engineering techniques. The term "article of manufacture" as used in this application encompasses a computer program accessible from any computer readable device, carrier or medium. For example, computer-readable media may include, but are not limited to, magnetic storage devices (eg, hard disks, floppy disks, or magnetic tapes, etc.), optical disks (eg, compact discs (CDs), digital versatile discs (DVDs) etc.), smart cards and flash memory devices (eg, erasable programmable read-only memory (EPROM), cards, stick or key drives, etc.). Additionally, various storage media described herein can represent one or more devices and/or other machine-readable media for storing information. The term "machine-readable medium" may include, but is not limited to, wireless channels and various other media capable of storing, containing, and/or carrying instructions and/or data.

FIG. 1 is a schematic diagram of a system to which the video segmentation method provided by this application can be applied. FIG. 1 shows a video conference system, which includes a conference control server 101 , a conference terminal 111 , a conference terminal 112 and a conference terminal 113 . The conference terminal 111 , the conference terminal 112 and the conference terminal 113 can establish a conference through the conference control server 101 .

A video conference usually includes at least two conference rooms. Each conference site can access the conference control server through a conference terminal. The conference terminal may be a device for accessing a video conference. The conference terminal can be used to receive conference data, and present conference content on the display device according to the conference data. The conference terminal may include a host and a display device. The host can receive conference data through a communication interface, generate a video signal according to the received conference data, and output the video signal to the display device in a wired or wireless manner. The display device presents the conference content according to the received video signal. Optionally, in some embodiments, the display device may be built in the host. For example, the conference terminal may be an electronic device with a built-in display device, such as a notebook computer, a tablet computer, and a smart phone. Optionally, in other embodiments, the display device may be a display device external to the host. For example, the host may be a computer host, and the display device may be a monitor, a television, or a projector. For another example, even if the host has a built-in display device, the display device used for presenting the meeting content may also be a display device external to the host. For example, the host may be a notebook computer, and the display device may be a display, a television or a projector externally connected to the notebook computer.

In some cases, a video conference may include a main venue and at least one branch venue. In this case, the conference terminal (for example, the conference terminal 111 ) in the main conference site can upload the collected media stream of the main conference site to the conference control server 101 . The conference control server 101 may generate conference data according to the received media stream, and send the conference data to conference terminals (eg, conference terminal 112 and conference terminal 113 ) in the sub-venue. The conference terminal 112 and the conference terminal 113 may present the conference content on the display device according to the received conference data.

In other cases, at least two conference sites in the video conference may not be divided into primary and secondary. The conference terminal in each conference site can upload the collected media stream to the conference control server 101 . For example, it is assumed that conference terminal 111 is the conference terminal used to access the video conference in site 1, conference terminal 112 is the conference terminal used to access the video conference in site 2, and conference terminal 113 is used to access the video conference in site 3 Conference terminal for the meeting. The conference terminal 111 can upload the collected media stream of the conference site 1 to the conference control server 101, and the conference control server 101 can generate conference data 1 according to the media stream of the conference site 1, and send the conference data 1 to the conference terminal 112 and the conference terminal. 113 , the conference terminal 112 and the conference terminal 113 may present the conference content on the display device according to the received conference data 1 . Similarly, the conference terminal 112 can also upload the collected media stream of the conference site 2 to the conference control server, and the conference control server 101 can generate conference data 2 according to the media stream of the conference site 2, and send the conference data 2 to the conference terminal 111 and the conference terminal 113, the conference terminal 111 and the conference terminal 113 can present the conference content on the display device according to the received conference data 2; the conference terminal 113 can also upload the collected media stream of the conference site 3 to the conference control server, and the conference control The server 101 can generate the conference data 3 according to the media stream of the conference site 3, and send the conference data 3 to the conference terminal 111 and the conference terminal 112, and the conference terminal 111 and the conference terminal 112 can present the conference data 3 on the display device according to the received conference data 3 meeting content.

Optionally, in some embodiments, the media stream may be an audio stream. Optionally, in other embodiments, the media stream may be a video stream. The media device responsible for collecting media streams may be built into the conference terminal (for example, a camera and a microphone in the conference terminal), or may be externally connected to the conference terminal, which is not limited in this embodiment of the present application.

Optionally, in some embodiments, the speaker of the conference uses the presentation during the speech. In this case, the media stream may be the audio stream of the speaker speaking. The presentation used by the speaker during the speech can be uploaded to the conference control server 101 through an auxiliary stream (also referred to as a data stream or a computer screen stream). The conference control server 101 generates conference data according to the received audio stream and auxiliary stream. Optionally, in some possible implementation manners, the conference data may include the received audio stream and auxiliary stream. Optionally, in some other possible implementation manners, the conference data may include a processed audio stream obtained by processing the received audio stream and the auxiliary stream. The processing of the received audio stream may be to perform a transcoding operation on the received audio stream, for example, the bit rate of the audio stream may be reduced, so as to reduce the amount of data required for transmitting the audio stream to other conference terminals. Optionally, in some other possible implementation manners, the conference data may include a received audio stream, an audio stream with a different bit rate from the received audio stream, and the auxiliary stream. In this way, the conference terminal can select an appropriate audio stream according to the network conditions and/or the way of accessing the conference. For example, if the network condition of the conference terminal is good or the conference is accessed through Wi-Fi, an audio stream with a higher bit rate can be selected, so that a clearer sound can be heard. For another example, if the network condition of the conference terminal is poor, an audio stream with a lower bit rate can be selected, which can reduce the occurrence of interruption of the conference live broadcast caused by the poor network condition. For another example, if the conference terminal uses a mobile network to access the conference, an audio stream with a lower bit rate can be selected, which can reduce traffic consumption. Optionally, in some other possible implementation manners, in addition to the audio stream and auxiliary stream of at least one bit rate, the conference data may also include subtitles corresponding to the speaker's speech. The subtitles may be generated by voice-to-text conversion of the speaker's speech based on speech recognition technology, or may be manually recorded by the speaker's speech, or may be generated based on speech-to-text conversion combined with manual modification of.

Optionally, in other embodiments, the media stream may be a video stream of the speaker during the speech. In other words, the media stream may include both the audio information and the picture information of the speaker during the speech. Correspondingly, the media stream uploaded to the conference control server 101 is the video stream. In some cases, it is assumed that the speaker uses a presentation during the presentation, and an output device (eg, projector, television, etc.) is used to present the presentation. The picture information in the media stream includes the presentation presented by the speaker. Therefore, the presentation is included in the video stream uploaded to the conference control server 101 . In this case, the conference control server 101 can directly determine the conference data according to the video stream. In other cases, the presentation used by the speaker during the speech may be uploaded to the conference control server 101 by way of auxiliary stream. The conference control server 101 can generate conference data according to the collected video stream and the auxiliary stream. Optionally, in some possible implementation manners, the conference data may include collected video streams and auxiliary streams. Optionally, in some other possible implementation manners, the conference data may include the processed video and the auxiliary stream obtained by processing the collected video stream. The processing of the collected video stream may be to perform a transcoding operation on the collected video stream, for example, the resolution of the video stream may be reduced, so as to reduce the amount of data required for transmitting the video stream to other conference terminals. Optionally, in some other possible implementation manners, the conference data may include a captured video stream, a video stream with a resolution different from that of the captured video stream, and the auxiliary stream. In this way, the conference terminal can select an appropriate video stream according to the network conditions and/or the way of accessing the conference. For example, if the network condition of the conference terminal is good or the conference is accessed through Wi-Fi, a video stream with a higher resolution can be selected, so that the audience can see a clearer picture. For another example, if the network condition of the conference terminal is poor, a video stream with a lower resolution can be selected, which can reduce the interruption of the live broadcast of the conference caused by the poor network condition. For another example, if the conference terminal uses a mobile network to access the conference, a video stream with a lower resolution can be selected, which can reduce traffic consumption. Optionally, in some other possible implementation manners, in addition to the video stream and auxiliary stream of at least one resolution, the conference data may also include subtitles corresponding to the speaker's speech. The subtitles may be generated by voice-to-text conversion of the speaker's speech based on speech recognition technology, or may be manually recorded by the speaker's speech, or may be generated based on speech-to-text conversion combined with manual modification of.

FIG. 2 is a schematic diagram of another system to which the video segmentation method provided by the present application can be applied. FIG. 2 shows a distance education system, which includes a course server 201 , a main device 211 , a client device 212 and a client device 213 .

The main device 211 can upload the captured media stream to the course server 201 . The course server 201 can generate course data according to the media stream, and send the course data to the client device 212 and the client device 213, and the client device 212 and the client device 213 can present on the display device according to the received course data Course content.

The main device 211 may be a notebook computer or a desktop computer. Client device 212 and client device 213 may be laptops, desktops, tablets, smartphones, and the like.

Optionally, in some embodiments, the teacher in charge of the lecture uses the presentation during the lecture. In this case, the media stream may be an audio stream of the teacher's lecture. The presentation used by the teacher in the course of the lecture can be uploaded to the course server 201 through the auxiliary stream. The course server 201 generates course data according to the received audio stream and auxiliary stream.

Optionally, in other embodiments, the media stream may be a video stream of a teacher during a lecture. In other words, the media stream may include both the audio information and the picture information of the teacher during the lecture. Correspondingly, the media stream uploaded to the course server 201 is the video stream. In some cases, it is assumed that the teacher uses the presentation during the lecture and uses an output device (eg, projector, television, etc.) to present the presentation. The picture information in the media stream includes the presentation presented by the teacher. Therefore, the presentation is included in the video stream uploaded to the course server 201 . In this case, the course server 201 can directly determine the course data according to the video stream. In other cases, the presentations used by the teacher in the course of lectures can be uploaded to the course server 201 by means of auxiliary streams. The course server 201 can generate course data according to the collected video stream and the auxiliary stream.

The specific content of the course data is similar to the specific content of the conference data, and will not be repeated for brevity.

FIG. 3 is a schematic flowchart of a video segmentation method provided according to an embodiment of the present application. The method shown in FIG. 3 may be performed by a video segmentation apparatus. The video segmentation apparatus may be a computer device capable of implementing the methods provided by the embodiments of the present application, such as a personal computer, a notebook computer, a tablet computer, a server, etc., or may be a computer device capable of implementing the methods provided by the embodiments of the present application. The internal hardware, for example, a graphics card, a graphics processing unit (Graphics Processing Unit, GPU), or, may also be a dedicated device for implementing the method provided in this embodiment of the present application. For example, in some embodiments, the video segmentation device may be the conference control server 101 in the system shown in FIG. 1 or a piece of hardware provided in the conference control server 101 . For another example, in other embodiments, the video segmentation device may be a conference terminal that uploads a media stream in the system as shown in FIG. 1 or a piece of hardware in the conference terminal. For another example, in other embodiments, the video segmentation apparatus may be the main device 211 in the system as shown in FIG. 2 or a piece of hardware provided in the main device 211 . For another example, in other embodiments, the video segmentation device may be the course server 201 in the embodiment shown in FIG. 2 or a piece of hardware in the course server 201 .

For ease of description, it is assumed that the method shown in FIG. 3 is applied in the system shown in FIG. 1 .

301. The video segmentation apparatus acquires text information of the video to be processed and voice information of the video to be processed, where the text information includes at least one of a presentation in the video to be processed and content description information of the video to be processed.

The presentation refers to the presentation presented by the speaker of the conference during the speaking process. The embodiment of the present application does not limit the file format of the presentation, as long as the presentation is displayed by the display device during the speech of the speaker, the presentation may be the presentation. For example, the presentation may be in ppt format or pptx format. As another example, the presentation may be a presentation in PDF format. For another example, the presentation may also be a document in word format or txt format.

The content description information is information that is uploaded by the speaker of the conference or the moderator of the conference and is used to describe the content of the speech before the conference starts. Optionally, in some embodiments, the content description information includes an outline, abstract and/or key information of the speech content of the speaker in the video conference. For example, the content description information may include keywords of the speech content of the speaker. For another example, the content description information may include a summary of the speech content of the speaker. For another example, the speech content of the speaker may include multiple parts, and the content description information may include the subject, abstract and/or keywords of each of the multiple parts.

The voice information may include voice-to-text conversion of the speaker's speech to obtain corresponding text. The specific implementation manner of speech-text conversion is not limited in the embodiments of the present application, as long as the recognized speech can be converted into corresponding text. The voice information may further include at least one pause point obtained by voice recognition of the speaker's speech. Pause points represent natural pauses in the speaker's speech.

302. The video segmentation apparatus determines a segmentation point of the to-be-processed video according to the text information and the voice information.

As described above, the textual information may include at least one of the presentation and the content description information. In other words, the text information can exist in the following three situations:

Case 1: Only the presentation is included in the text information;

Case 2: Only the content description information is included in the text information;

Case 3: The text information includes the presentation and the content description information.

In other words, in some cases, the speaker may only present the presentation during the speech without uploading the content description information in advance. Therefore, the above situation 1 may occur. In other cases, the speaker may simply upload the description in advance without showing the presentation during the presentation. Therefore, the above situation 2 may occur. In other cases, the speaker may present the presentation while speaking and upload the description in advance. Therefore, it is possible to write Case 3 above.

For the above case 1, the video segmentation apparatus may determine the segmentation point of the video to be processed according to the presentation.

For the above case 2, the video segmentation apparatus may determine the segmentation point of the video to be processed according to the content description information.

Optionally, in some embodiments, for the above case 3, the video segmentation apparatus may determine the segmentation point of the video to be processed according to one of the presentation and the content description information. In other words, when the text information includes the presentation and the content description information, the video segmentation apparatus may determine the segmentation point of the video to be processed according to the presentation or the content description information.

Optionally, in some embodiments, when the text information includes the presentation and the content description information, the video segmentation device may determine the presentation duration of the current page of the presentation, and perform a presentation according to the presentation duration. The presentation duration of the current page determines whether to determine the segment point of the to-be-processed video according to the presentation, or to determine the segment point of the to-be-processed video according to the content description information.

Optionally, in some embodiments, the video segmentation apparatus may determine the pending processing according to the content description information and the voice information when the presentation of the current page of the presentation is longer than the first preset duration. Segmentation point of the video. In this way, it can be avoided that one segment of the video is too long due to the speaker demonstrating the same content for a long time. The first preset duration can be set as required. For example, the first preset time period may be 10 minutes. For another example, the first preset duration may be 15 minutes.

Optionally, in some embodiments, the video segmentation apparatus may determine, according to the content description information and the voice information, when the presentation duration of the current page of the presentation is less than or equal to the second preset duration. The segmentation point of the video to be processed. In this way, it can be avoided that one segment of the video is too short due to frequent switching of the displayed content of the presentation by the speaker. Similar to the first preset duration, the second preset duration can be set as required. For example, the second preset duration may be 20 seconds. For another example, the second preset duration may be 10 seconds.

The first preset duration is greater than the second preset duration.

Optionally, in some embodiments, the video segmentation apparatus may, when the presentation duration of the current page of the presentation is greater than the second preset duration and less than or equal to the first preset duration, according to the presentation. The text and the voice information are used to determine the segmentation point of the video to be processed.

Optionally, in other embodiments, only the first preset duration may be set. If the presentation duration of the presentation on the current page is greater than the first preset duration, the segment point of the video to be processed is determined according to the content description information and the voice information. If the presentation duration of the current page of the presentation is not greater than the first preset duration, the segment point of the video to be processed may be determined according to the presentation and the voice information. The presentation duration of the current page of the presentation is how long the presentation stays on the current page.

Optionally, in some embodiments, the start moment of the presentation duration of the current page of the presentation is the moment when the presentation is switched to the current page, and the end moment of the presentation duration of the current page of the presentation is the moment when the presentation changes from the current page. The moment when a page switches to another page.

For example, if the presentation is switched to the nth page at time T1 ( _n is a positive integer greater than or equal to ₁ ), the video segmentation device can start timing from time T1. If the presentation has not been switched to the n+1th page when the timing exceeds the first preset time, the video segmentation device may determine the to-be-processed video according to the content description information and the voice information segmentation point. If the presentation is switched to the n _{+ 1th} page at time T2 (T2 is greater _than T1), and the duration from time T1 to time T2 is less _than or equal to the _second preset duration, the video segmentation device The segmentation point of the video to be processed can be determined according to the content description information and the voice information. If the duration from time T1 to time T2 is less _than or equal to the first preset duration and greater than the _second preset duration, the video segmentation device can determine the video to be processed according to the presentation and the voice information segmentation point. More specifically, the video segmentation device may determine the segmentation point of the to-be-processed video according to the presentation on the nth page and the voice information.

Optionally, in some other embodiments, the start moment of the presentation duration of the current page of the presentation may be the point of the previous segment, and the end moment of the presentation duration of the current page of the presentation is when the presentation starts from the current page. The moment to switch to another page.

For example, it is assumed that the presentation is switched to the nth page at time T3 ₍ n is a positive integer greater than or equal to 1), and the presentation duration on the nth page is longer than the first preset duration. In this case, the video segmentation device determines, according to the content description information and the voice information, a segmentation point of the to-be _- processed video as time T4. _The video segmentation means may start timing from time T4. If the presentation has not been switched to the n+1th page when the timing exceeds the first preset time, the video segmentation device may determine the to-be-processed video according to the content description information and the voice information segmentation point. If the presentation is switched to the n _{+ 1th} page at time T5 ₍ T5 is greater than T4 ₎ , and the duration from time _T4 to time T5 is not greater than the first preset duration and greater than the second preset duration , the video segmentation device can determine the segmentation point of the video to be processed according to the presentation and the voice information. More specifically, the video segmentation device may determine the segmentation point of the to-be-processed video according to the presentation on the nth page and the voice information.

Optionally, in some other embodiments, when the text information includes the presentation and the content description information, the video segmentation device may determine the segmentation of the video to be processed according to the presentation and the voice information. paragraph point. In other words, even if the text information includes both the presentation and the content description information, the video segmentation apparatus may only refer to the presentation and the voice information (that is, without using the content description information) to determine the video to be processed segmentation point.

Optionally, in other embodiments, in the case that the text information includes the presentation and the content description information, the video segmentation device may determine, according to the content description information and the voice information, the segment point. In other words, even if the text information includes both the presentation and the content description information, the video segmentation apparatus can only refer to the content description information and the voice information (that is, the presentation document will not be used) to determine the video to be processed segmentation point.

The video segmentation device determining the segmentation point of the to-be-processed video according to the presentation and the voice information may include: the video segmentation device determines a switching point of the presentation, and the content presented in the presentation before and after the switching point different; the video segmentation device determines at least one pause point according to the voice information; the video segmentation device determines the segmentation point according to the switching point and the at least one pause point.

The switching point of a presentation is the moment when the presentation switches. A presentation switching occurs may refer to a page turning of the presentation. For example switching from page 1 to page 2. The presentation switching can also mean that the content of the presentation changes without turning pages. For example, where the presentation is a text document, the speaker may show only a portion of a page of the presentation (eg, the top half), and then scroll to the remainder of the page (eg, the bottom half). Although the presentation is not a page turn at this point, the content in the presentation has changed.

Optionally, in some embodiments, the video segmentation apparatus may acquire a switching signal for instructing to switch the content of the presentation. In this case, the video segmentation apparatus may determine that the moment when the switching signal is acquired is the switching point.

Optionally, in some embodiments, the video segmentation apparatus may acquire the content of the presentation. In this case, the video segmentation means may determine the switching point according to the change in the content of the presentation. For example, the video segmentation apparatus may determine that the first moment is the switching when it is determined that the content of the presentation presented at the first moment of the video to be processed is different from the content of the presentation presented at the second moment point. Optionally, in some embodiments, the first moment and the second moment are adjacent moments, and the first moment is before the second moment. Optionally, in other embodiments, the first moment is before the second moment and the interval between the first moment and the second moment is less than a preset duration. In other words, in this case, the video segmentation device can detect whether the content presented in the presentation changes at intervals of a certain period of time.

Optionally, in some embodiments, the video segmentation apparatus may determine the switching point in combination with the acquired switching signal for instructing switching the content of the presentation and the content presented by the presentation. For example, the video segmentation device _acquires the switching signal at time T1. _The video segmentation apparatus can obtain the content presented by the F1 frame before the presentation at time T1 and the content presented by the F2 frame after the time T1, where _F1 and _F2 are positive _integers greater _than or equal to ₁ . Optionally, in some embodiments, F ₁ and F ₂ may take smaller values, for example, F ₁ and F ₂ may be equal to 2. This reduces the amount of computation. If the content presented in _two consecutive frames in the F1 frame and the F2 frame _of the presentation is different, it can be determined that the moment at which the presented content of the presentation changes is the switching point. For example, suppose that _both F1 and F2 have values of ₂ . If the content presented in the second frame and the third frame of the four frames of the presentation are different, it can be determined that the moment at which the second frame is located is the switching point. Using the switching signal and the content presented in the presentation to determine the switching point can avoid the situation that the determined switching point is inaccurate due to the non-synchronization of the switching signal and the screen switching of the presentation.

Optionally, in some embodiments, the video segmentation apparatus may determine whether the content presented in the presentation at different moments (or different frames) is the same according to the following manner: the video segmentation apparatus compares the presentation at different moments (or The change of the pixel value at the same position in different frames) exceeds the number P of preset change values. If P is greater than the first preset threshold P ₁ , the content presented by the video segmentation device in the presentation has changed. Optionally, in some embodiments, the change of the pixel value may be determined by calculating the absolute value of the difference of the pixel gray value. Optionally, in other embodiments, the change in pixel value may be determined by calculating the sum of absolute values of the differences in the three color channels.

Optionally, in some embodiments, if P is greater than the second preset threshold P ₂ (P ₂ is less than P ₁ ), the video segmentation apparatus may determine the keyword according to the subsequent presentation. For example, the video segmentation device determines that the number of pixel values in the presentation at time T1 and the presentation at time T2 ₍ time T2 is later _than time T1 ₎ at the same position where the change in pixel value exceeds the preset change value is greater _than P ₂ and less than P ₁ . _In this case, the video segmentation apparatus may determine the keywords according to the presentation at time T2.

As mentioned above, the voice information may further include at least one pause point. Optionally, in some embodiments, the at least one pause point used to determine the segment point may be all pause points from the start time to the current time. If the segment point determined in step 302 is the first segment point of the video to be processed, the start time is the start time of the video to be processed. If the segment point determined in step 302 is the kth segment point of the video to be processed (k is a positive integer greater than or equal to 2), then the start time is the time at which the k-1th segment point is located. Optionally, in other embodiments, the video segmentation apparatus may further determine a pause point within a time range according to the moment at which the switch point is located, and the switch point is within this time range. For example, if the switching point is at time T1, the video segmentation apparatus can determine the pause point from time T1 _{- t} to time T1 ₊ t.

The video segmentation apparatus determines that the switching point is the segmentation point in the case that the switching point is determined to be the same as one of the at least one pause points. In the case where the video segmentation device determines that the switching point is different from any one of the at least one pause point, the video segmentation device determines a pause point that is closest to the switch point in the at least one pause point as the segmentation point . The distance between the pause point and the switch point refers to the time difference between the pause point and the switch point. For example, assuming that the switching point is located at time T1, _one of the at least one pause point is located at time _T2 , and the difference between _T2 and T1 is _t . Assuming that the difference between the at least _one pause point except the pause point at time T2 and the time T1 is greater _than t, the _pause point at time T2 is the segment point. If the distances from two of the at least one pause point to the switch point are the same and smaller than the distances from the other pause points except the two pause points to the switch point, then any one of the two pause points can be determined. A pause point is the switch point.

The video segmentation device determining the segmentation point of the video to be processed according to the content description information and the voice information may include: the video segmentation device according to the voice information, the keywords of the content description information, and the voice information. Pause point, to determine the segmentation point of the video to be processed.

Optionally, in some possible implementations, the video to be processed may be divided into multiple segments of voice information. The first speech information segment and the second speech information segment are two consecutive speech information segments among the plurality of speech information segments. The first piece of voice information follows the second piece of voice information. The video segmentation device can determine a first segmentation point according to the first voice information segment, the second voice information segment, the keywords of the content description information and the pause point in the voice information, the first segmentation point is one of at least one segment point included in the video to be processed.

The video segmentation device can cut out text fields on the voice information with a window length W and a step size S. The video segmentation device can cut out at least one text field with a length of W. Each text field of length W is a segment of speech information.

The video segmentation apparatus may determine whether the first segment of speech information is similar to the second segment of speech information. If the first voice information fragment is not similar to the second voice information fragment, it may be determined that a segment point of the video to be processed is near the first voice information fragment. If the second voice information fragment is similar to the first voice information fragment, continue to determine the third voice information fragment adjacent to the first voice information fragment and located after the first voice information fragment and the first voice information fragment Is it similar.

The similarity may be used as a criterion for evaluating whether the first voice information segment is similar to the second voice information segment. If the similarity between the first voice information fragment and the second voice information fragment is greater than or equal to a similarity threshold, it can be considered that the first voice information fragment is similar to the second voice information fragment; if the first voice information fragment and the second voice information fragment are similar If the similarity of the second voice information segment is smaller than the similarity threshold, it can be considered that the first voice information segment and the second voice information segment are not similar.

Optionally, in some possible implementation manners, the video segmentation device may be based on the keywords of the first voice information fragment, the keywords of the second voice information fragment, the content of the first voice information fragment, the first voice information fragment. The content of the two pieces of speech information and the keywords of the content description information determine the similarity between the first piece of speech information and the second piece of speech information.

The video segmentation device may determine the keyword of the first voice information segment. Assuming that the number of keywords determined from the first voice information segment is N, the number of keywords determined from the content description information is M, and there are no repeated keywords among the M keywords and the N keywords.

The video segmentation device may determine keywords according to the following methods:

Step 1, according to a preset stop word list or according to the part of speech of each word in the text, remove words that do not represent actual meaning, such as "of", "this", "then" and so on. Stop words (Stop Words) are some words or words that are manually input and not generated automatically. These words do not represent actual meaning and are filtered out before or after processing the natural language data. A set of stop words consisting of stop words can be called a stop word list.

Step 2, count the frequency of occurrence of each word in the text in the remaining words. The frequency of occurrence of each word in the text can be determined according to the following formula:

TF(n)=N(n)/All_N, Equation 1.1

Among them, TF(n) represents the frequency of occurrence of the nth word in the remaining words of the text after step 1, N(n) represents the number of occurrences of the nth word, and All_N represents the remaining words the total number of words.

Step 3: Determine at least one word with the highest frequency of occurrence as a keyword of the text.

For example, if the text is content description information, the M words with the highest frequency may be determined as keywords of the content description information, where M is a positive integer greater than or equal to 1. If the text is the first voice information segment, it can be determined that N words with the highest frequency of occurrence are keywords of the first voice information segment, and N is a positive integer greater than or equal to 1. If it is determined that the frequency of occurrence of one or more words in the N words is the same as the keyword of the content description information, the repeated words in the N words are deleted, and the following words are selected as the The keyword of the first voice information segment. For example, assuming that N is equal to 2 and M is equal to 1, the keyword of the content description information includes "student". Assuming that the determined word with the highest frequency in the first voice information segment is "student", then continue to determine the word with the second highest frequency. If the word with the second highest frequency is "school", it can be determined that "school" is a keyword of the first voice information segment, and the word with the third highest frequency is continued to be determined. Assuming that the word with the third highest frequency is "course", it can be determined that "course" is another keyword of the first voice information segment. If the text is the second voice information segment, it can be determined that the N words with the highest frequency are keywords of the second voice information segment, and N is a positive integer greater than or equal to 1. If it is determined that the frequency of occurrence of one or more words in the N words in the second speech information segment is the same as the keyword of the content description information, then the repeated words in the N words are deleted, and the following words are selected as the The keyword of the second voice information segment.

Optionally, in some embodiments, the video segmentation apparatus may determine the first keyword vector according to the keyword of the first voice information segment, the keyword of the content description information, and the content of the first voice information segment. . Specifically, the video segmentation device can determine the frequency of the keyword of the first voice information segment and the keyword of the content description information appearing in the content of the first voice information segment, and the frequency is the first keyword vector . The content of the speech information piece refers to all words included in the speech information piece. For example, it is assumed that the keyword of the content description information is "student", and the keywords of the first voice information segment are "course" and "school". Assuming that the frequencies of the above three keywords appearing in the first voice information segment are 0.1, 0.2, and 0.3, respectively, the first keyword vector is (0.3, 0.2, 0.1).

Similarly, the video segmentation apparatus may also determine the second keyword vector according to the keyword of the second voice information segment, the keyword of the content description information, and the content of the second voice information segment. Specifically, the video segmentation device can determine the frequency of the keyword of the second voice information segment and the keyword of the content description information appearing in the content of the second voice information segment, and the frequency is the second keyword vector . For example, it is assumed that the keyword of the content description information is "student", and the keywords of the first voice information segment are "breakfast" and "nutrition". Assuming that the frequencies of the above three keywords appearing in the second voice information segment are 0.3, 0.25, and 0.05, respectively, the second keyword vector is (0.3, 0.25, 0.05).

The video segmentation device determines the distance according to the first keyword vector and the second keyword vector. If the distance is greater than the preset distance, it can be considered that the similarity between the first voice information segment and the second voice information segment is similar. less than the similarity threshold. In this case, the video segmentation device determines the segmentation point according to the first voice information segment.

The video segmentation device may determine a distance according to the first keyword vector and the second keyword vector in the following manner:

Step 1: Expand the first keyword vector into a first vector, and expand the second keyword vector into a second vector, wherein the keywords corresponding to the first vector and the keywords corresponding to the second vector include the first keyword vector. A keyword of a voice information fragment, a keyword of the second voice information fragment and a keyword of the content description information, and the keywords corresponding to the first vector have no repeated keywords, and the keywords corresponding to the second vector There are no duplicate keywords in .

For example, assuming that the first keyword vector is (0.3, 0.2, 0.1), the corresponding keywords are "school", "course" and "student", assuming that the second keyword vector is (0.3, 0.25, 0.05) , the corresponding keywords are "students", "breakfast" and "nutrition". In this case, the first vector is (0.3, 0.1, 0, 0.2, 0), the corresponding keywords are "school", "student", "breakfast", "course", "nutrition", the second The vector is (0, 0.3, 0.25, 0, 0.05), and the corresponding keywords are "school", "student", "breakfast", "course", "nutrition".

Step 2: Calculate the distance between the first vector and the second vector. The distance between the first vector and the second vector is the distance determined according to the first keyword vector and the second keyword vector.

Optionally, in some embodiments, the distance between the first vector and the second vector may be Euclidean distance. Because the same keywords in the two voice information fragments before and after may be rare. Therefore, if the distance between the first vector and the second vector is a cosine distance, there may be many 0 values in the calculation result. Therefore, it may be more appropriate to choose the Euclidean distance as the distance between the first vector and the second vector.

Optionally, in other embodiments, the distance between the first vector and the second vector may be a cosine distance.

In addition to using word frequency vectors of two adjacent speech information segments to determine whether two speech information segments are similar, other methods may also be used to determine whether two speech information segments are similar.

For example, the first keyword vector and the second keyword vector may also be word frequency-inverse document frequency, binary word frequency, or the like. Determining the distance between the first keyword vector and the second keyword vector may be determining an n-norm distance between the first keyword vector and the second keyword vector (n is a positive integer greater than or equal to 1), A relative entropy distance of the first keyword vector and the second keyword vector is determined.

Taking the above-mentioned first vector (ie (0.3, 0.1, 0, 0.2, 0)) and second vector (ie (0, 0.3, 0.25, 0, 0.05)) as an example, the first vector and the Binarize the two vectors. The first vector after binarization is (1,1,0,1,0), and the second vector after binarization is (0,1,1,0,1). Then, the 1-norm distance is calculated to obtain the repetition degree of the keywords of the first voice information segment and the keywords of the second voice information segment. Keyword repetition can be thought of as a special form of distance. Whether the first voice information segment and the second voice information segment are similar may be determined by using the repetition degree of the keywords. If the repetition degree of the keyword is greater than or equal to a preset repetition degree, it can be considered that the first voice information fragment and the second voice information fragment are similar; if the repetition degree of the keyword is less than the preset repetition degree, it can be considered that The first voice information segment and the second voice information segment are not similar. It can be seen that the preset repetition degree can be regarded as the similarity threshold in this case.

Optionally, in other embodiments, the extraction of keywords may also be determined according to word frequency-inverse document frequency. The word frequency can be determined based on Equation 1.1. The inverse document frequency can be determined according to the following formula:

IDF(n)=log(Num_Doc/(Doc(n)+1), Equation 1.2

where IDF(n) represents the inverse document frequency of the nth word, Num_Doc represents the total number of documents in the corpus, and Doc(n) represents the number of documents in the corpus that contain the nth word.

Term Frequency - Inverse Document Frequency can be determined according to the following formula:

TF-IDF(n)=TF(n)×IDF(n), formula 1.3

where TF-IDF(n) represents the term frequency-inverse document frequency of the nth word. If the keyword is determined according to the word frequency-inverse document frequency, the first keyword vector is composed of the word frequency of the keyword-inverse document frequency.

When determining keywords according to word frequency-inverse document frequency, it may not be necessary to remove meaningless words first.

Optionally, in other embodiments, the extraction of keywords may also be based on a word graph text ranking (TextRank) method. If the keywords are determined according to TextRank based on the word graph, the first keyword vector may be composed of the weights of the words.

In the case that the first voice information fragment and the second voice information fragment are not similar, the video segmentation apparatus may determine the segmentation point according to the first voice information fragment.

The video segmentation apparatus may first determine whether a pause point is included in the first voice information segment. If the first voice information segment includes a pause point, it can be determined that the pause point is the segment point. If the first voice information segment includes multiple pause points, it can be determined whether the word after each pause point in the multiple pause points is a preset word. The presupposition includes conjunctions with segmental meanings, such as "next", "below", "next point" and the like. The words after the pause point refer to words that are located after the pause point and adjacent to the pause point. If the word after only one of the plurality of pause points is a preset word, it can be determined that the pause point is a segmentation point. If words following at least two of the plurality of pause points are preset words, it may be determined that the pause point with the pause duration among the at least two pause points is the segment point. If the words following the multiple pause points are not the preset words, it may be determined that the pause point with the longest pause duration among the multiple pause points is the segment point. If the first speech information segment does not include a pause point, the segment point may be determined according to the pause point adjacent to the first speech segment. It can be understood that there may be two pause points adjacent to the first speech segment, one is located before the first speech segment, and the other is located after the first speech segment. The video segmentation device may determine the segmentation point according to the distance between the two pause points and the first voice information segment. If the pause point is before the first voice information segment, the distance between the pause point and the first voice information segment may be the number of words or the time difference between the pause point and the starting position of the first voice information segment. If the pause point is after the first voice information segment, the distance from the pause point to the first voice information segment may be the number of words or the time difference between the pause point and the end position of the first voice information segment. For the convenience of description, the pause point adjacent to the first speech information segment before the first speech information segment is referred to as the front pause point, and the distance between the former pause point and the first speech information segment is referred to as Distance 1; the pause point adjacent to the first speech segment after the first speech segment is called a post-pause point, and the distance from the post-pause point to the front of the first speech information segment is called a distance 2. If the distance 1 is less than the distance 2, it can be determined that the front pause point is the segment point; if the distance 1 is greater than the distance 2, the rear pause point can be determined to be the segment point. If the distance 1 is equal to the distance 2, the word after the front pause point (hereinafter referred to as word 1) and the word after the rear pause point (hereinafter referred to as word 2) can be determined; if word 1 is the preset word and word 2 is not the preset word Assuming a word, then determine that the previous pause point is the segment point; if word 1 is not the preset word and word 2 is the preset word, then determine the rear pause point as the segment point; if word 1 and word 2 are the segment point If both are the preset words or are not the preset words, it can be determined that the segment point with the longest pause time among the front pause point and the rear pause point is the segment point.

As mentioned above, the pause point is the speaker's natural pause. Therefore, the pause point has a certain length of time. Optionally, in some embodiments, if it is determined that the pause point is a segment point, the middle moment of the pause point may be determined to be the segment point. Optionally, in other embodiments, if it is determined that the pause point is a segment point, it may be determined that the end time of the pause point is the segment point. Optionally, in other embodiments, if it is determined that the pause point is a segment point, it may be determined that the start time of the pause point is the segment point.

303. The video segmentation device segments the to-be-processed video according to the segmentation point.

If the segment point is the first segment point of the video to be processed, the start time of the segment is the start time of the video to be processed, and the end time of the segment is the segment point. If the segment point is the kth segment of the video to be processed (k is a positive integer greater than or equal to 2), then the start time of the segment is the k-1th segment point, and the The end time is the segment point.

After the segment is determined, the video segmentation device may also determine a digest of the segment.

304. The video segmentation apparatus may determine an abstract of the segment according to the content of the segmented speech information, the keywords of the segmented speech information, and the keywords of the target text. The target text includes at least one of the presentation and the content description information.

Optionally, in some embodiments, the video segmentation apparatus may first determine a third keyword vector, and then determine an abstract of the segment according to the third keyword vector.

The video segmentation device may determine a third keyword vector according to the content of the segmented speech information, the keywords of the segmented speech information and the keywords of the target text, wherein the content of the segmented speech information refers to the composition of the segmented speech information. The entire sentence of the speech information of the segment.

It can be understood that if the text information only includes the presentation, the target text includes the presentation; if the text information only includes the content description, the target text includes the content description; if the text If the information includes the presentation and the content description information, the target text includes the presentation and the content description.

The implementation manner of the video segmentation device determining the keywords of the segmented voice information, the implementation manner of the video segmentation device determining the keywords of the target text, and the video segmentation device determining the keywords of the first voice information segment is implemented similarly.

Optionally, in some embodiments, if the video segmentation device compares the number P of the pixel values of the presentation at different times (or different frames) at the same position with changes exceeding the preset change value, the number P is greater than the second preset value. The threshold value P ₂ (P ₂ is smaller than P ₁ ), the video segmentation apparatus can determine the keyword of the target text according to the following presentation. For example, the video segmentation device determines that the number of pixel values in the presentation at time T1 and the presentation at time T2 ₍ time T2 is later _than time T1 ₎ at the same position where the change in pixel value exceeds the preset change value is greater _than P ₂ and less than P ₁ . _In this case, the video segmentation device may determine the keywords of the target text according to the presentation at time T2.

For example, suppose the number of keywords determined from the presentation is L, the number of keywords determined from the content description information is M, the number of keywords determined from the segmented voice information is Q, the L keywords , there are no repeated keywords among the M keywords and the Q keywords.

Specifically, the video segmentation apparatus may first determine M keywords from the content description information, and then determine L words that appear most frequently in the presentation. If one or more words in the L words also belong to the M keywords, delete the one or more words from the L words, and then continue to determine the word with the second highest frequency from the presentation , until the determined L keywords do not intersect with the M keywords. After that, the video segmentation device determines Q words from the segmented speech information. If one or more words in the Q words belong to the M keywords or the L keywords, delete the one or more words from the Q words, and then continue to delete the one or more words from the segmented speech information Determine the word with the second highest frequency until the determined Q keywords have no intersection with the L keywords and the M keywords.

The third keyword vector includes the Q keywords, the L keywords, and the frequency of occurrence of the M keywords in the segmented speech information. It can be understood that if the target text does not include the content description information, the value of M is 0; if the target text does not include the presentation, the value of L is 0.

The video segmentation apparatus may determine an abstract of the segment according to the determined third keyword vector.

Specifically, the video segmentation device can determine a reference text according to the content of the target text and the segmented voice information, where the reference text includes J sentences, and J is a positive integer greater than or equal to 1; according to the segmented voice The keywords of the information, the keywords of the target text, and each of the J sentences, determine J keyword vectors; according to the third keyword vector and the J keyword vectors, determine the abstract of the segment . The jth keyword vector in the J keyword vectors is the frequency of the keywords of the segmented speech information and the keywords of the target text appearing in the jth sentence.

In the case that the target text includes redundant sentences, delete the redundant sentences in the target text, obtain a revised target text, and combine the revised target text with the content of the segmented speech information to obtain the reference text; if the target text does not include the redundant sentence, combine the target text with the content of the segmented speech information to obtain the reference text. In other words, where the target text includes the presentation and the content description information, one or more sentences in the presentation may also appear in the content description information. In this case, delete one or more sentences in the presentation that are the same as the content description information, and then combine the presentation with the redundant sentences deleted, the content description information and the content of the segmented speech information to obtain the reference text. If the target text does not include redundant sentences, for example, any sentence in the presentation appears in the content description information, or the target text only includes one of the presentation and the content description information, Then, the target text can be directly combined with the content of the segmented voice information to obtain the reference text.

The video segmentation device determines the abstract of the segment according to the third keyword vector and the J keyword vectors, including: the video segmentation device determines according to the third keyword vector and the J keyword vectors. J distances, wherein the jth distance among the J distances is determined according to the third keyword vector and the jth keyword vector among the J keyword vectors, and j is greater than or equal to 1 and less than or A positive integer equal to J; determine the R distances with the shortest distances among the J distances, where R is a positive integer greater than or equal to 1 and less than J; determine the summary of the segment, wherein the summary of the segment includes and the R distances distance to the corresponding sentence. The specific implementation of the video segmentation device determining the jth distance according to the third keyword vector and the jth keyword vector is the same as the video segmentation device determining the distance according to the first keyword vector and the second keyword vector. The implementation is similar, the difference is: the jth distance determined according to the third keyword vector and the jth keyword vector is the Euclidean distance; the distance determined according to the first keyword vector and the second keyword vector It can be Euclidean distance or cosine distance. The reason why the jth distance determined between the third keyword vector and the jth keyword vector cannot be the cosine distance is that the jth keyword vector is normalized when calculating the cosine distance. But the modulo length of the jth tube to detect your vector just reflects the overall frequency of sentence j and keywords, so it cannot be normalized.

The above-mentioned vectors (eg, the first keyword vector, the second keyword vector, the third keyword vector, and the jth keyword vector) are all the frequencies (ie, word frequencies) of keywords appearing in a specific text. In other embodiments, the above-mentioned vector may also be determined according to a word vector determined by a word to vector (word to vector, word2vec). For example, the first keyword vector can be determined by the following steps: determining the word vector of each keyword by using word2vex; adding the word vectors of all keywords and taking an average to obtain the first keyword vector. The manner of determining the second keyword vector is similar to that of the first keyword vector, and details are not described here. For another example, the third keyword vector may be determined by the following steps: determining the word vector of each keyword by using word2vex; determining the word frequency of each keyword; and weighting the word vectors of all keywords according to the word frequency of each keyword. Averaged to obtain the third keyword vector. For another example, the jth keyword vector can be determined by the following steps: perform word segmentation on the jth sentence and remove stop words; use word2vex to determine the word vector of each remaining word; add all word vectors and average them, Get the jth keyword vector. In the case where the keyword vector is determined based on word2vex, the direct distance between the third keyword vector and the jth keyword vector may be a cosine distance.

FIG. 4 is a schematic diagram of a conference process provided according to an embodiment of the present application.

401, the conference terminal 1 transmits the audio and video stream 1 to the conference control server.

402, the conference terminal 2 transmits the audio and video stream 2 to the conference control server.

403, the conference terminal 3 transmits the audio and video stream 3 to the conference control server.

404. The conference control server determines the main conference site.

It is assumed that the main conference site determined by the conference control server is the conference site where the conference terminal 1 is located.

405 , the conference control server sends the conference data to the conference terminal 2 and the conference terminal 3 .

406, the conference terminal 2 and the conference terminal 3 store the conference data.

Optionally, in some embodiments, the conference control server may also send the conference data to the conference terminal 1, and the conference terminal 1 may also store the conference data.

407 , the conference control server segments the audio and video stream 1 in real time (ie, determines segment points), and extracts a summary of each segment.

408 , the conference control server sends the segment point and the summary to the conference terminal 2 and the conference terminal 3 . In this way, the conference terminal 2 and the conference terminal 3 can independently select a review point to play back the video. Of course, in some implementations, the conference control server may also send the segment points and the summary to the conference terminal 1 .

FIG. 5 is a schematic flowchart of a video segmentation method provided according to an embodiment of the present application.

501. The video segmentation apparatus determines whether the conference reservation includes text related to the conference content. In other words, the video segmentation apparatus can determine whether the to-be-processed video includes content description information. If the determination result is yes (that is, the to-be-processed video includes content description information), step 502 is performed; if the determination result is no (that is, the to-be-processed video does not include content description information), step 503 is performed.

502. The video segmentation apparatus extracts keywords of texts related to the conference content. In other words, the video segmentation apparatus determines the keywords of the content description information.

After the keyword of the content description information is determined, step 503 may be performed.

503. The video segmentation apparatus determines whether there is a screen presentation presentation in the video to be processed. In other words, the video segmentation device can determine whether the video to be processed includes a presentation, and the presentation is presented on a screen. If the determination result is yes (that is, the to-be-processed video includes a presentation), step 504 is executed. If the determination result is no (that is, the video to be processed does not include a presentation), then step 505 is performed. In 504, the video segmentation apparatus determines the position of the screen for displaying the presentation. After the video segmentation apparatus determines the position of the screen, step 506 may be performed.

505. The video segmentation apparatus determines whether there is a presentation delivered through the auxiliary stream. In other words, in some possible implementations, the conference speaker may not present the presentation through the screen, but upload the presentation to the conference control server through an auxiliary stream. The conference terminals in other conference sites can obtain the presentation used by the conference speaker during the speaking process according to the auxiliary stream. If the determination result is yes (that is, there is a presentation transmitted through the auxiliary stream), step 506 is executed. If the determination result is no (that is, there is no presentation transmitted through the auxiliary stream), the segmentation point of the video to be processed may be determined according to the voice information.

506. The video segmentation apparatus determines whether the duration from the last segment point to the current moment exceeds the first preset duration. If the video segmentation apparatus determines that the duration from the last segment point to the current moment is greater than the first preset duration (ie, the determination result is yes), step 507 is executed. If the video segmentation device determines that the duration from the last segment point to the current moment is not greater than the first preset duration, step 508 is executed. It can be understood that, if the segmentation point determined by the video segmentation device is the first segmentation point, the previous segmentation point refers to the start moment of the video to be processed. For the convenience of description, the duration from the point of the shirt segment to the current moment may be referred to as the presentation duration.

507. The video segmentation apparatus determines a segmentation point of the to-be-processed video according to the content description information and the voice information.

508. The video segmentation apparatus determines a segmentation point of the video to be processed according to the presentation and the voice information. The specific implementation manner of the video segmentation device determining the segmentation point of the to-be-processed video according to the presentation and the voice information may refer to the embodiment shown in FIG. 3 , which is unnecessary to describe here.

After the video segmentation device determines the segmentation point of the video to be processed, step 509 and step 510 may be performed.

509. The video segmentation apparatus determines the segmented voice information and a keyword of the segmented voice information, where the segmented voice information is the voice information between the previous segmented point of the segmented point and the segmented point. It can be understood that, if the segment point is the first segment point of the video to be processed, the segment voice information is the voice information between the start time of the to-be-processed video and the segment point.

510. The video segmentation apparatus determines a segment abstract according to the segmented voice information, the keywords of the segmented voice information, and the keywords of the target text. For the specific implementation manner of steps 509 and 510, reference may be made to the embodiment shown in FIG. 3 , which is not repeated here.

It can be understood that, in some other possible implementation manners, in the process of segmenting the video and extracting the abstract, the video segmentation device may first determine whether there is a presentation displayed on the screen in the video to be processed, and then Determine whether the conference reservation includes text related to the conference content, and finally determine whether the presentation is transmitted through the auxiliary stream. In some other possible implementation manners, the video segmentation device may also first determine whether there is a presentation delivered through an auxiliary stream, then determine whether the conference reservation includes text related to the conference content, and finally determine whether there is any text in the to-be-processed video. Presentation on screen.

The following will describe how the video segmentation apparatus determines the segmentation point of the video to be processed according to the content description information and the voice information with reference to FIG. 6 . In addition, the implementation manner of how the video segmentation apparatus determines the segmentation point of the to-be-processed video according to the voice information can also be referred to FIG. 6 .

FIG. 6 is a schematic flowchart of a method for segmenting a video according to an embodiment of the present application.

601 , the video segmentation apparatus continues to intercept voice information segments from the voice information with the window length W and the step size S.

602, the video segmentation apparatus extracts the keywords of each voice information segment. Specifically, the video segmentation device extracts N keywords from each voice information segment.

If the video segmentation device has extracted the keywords of the content description information, then step 603 may be performed after step 602; if the video segmentation device has not extracted the keywords of the content description information, then step 604 may be performed after step 602. . The video segmentation device has extracted the keywords of the content description information, which means that the video segmentation device determines that the to-be-processed video includes content description information. In this case, the segmentation point determined by the video segmentation device is determined according to the content description information and the voice information. The video segmentation device has not extracted the keywords of the content description information, which means that the video segmentation device determines that the video to be processed does not include content description information. In this case, the segmentation point determined by the video segmentation device is determined according to the voice information.

603. The video segmentation apparatus determines the keyword in the ith voice information segment and the word frequency vector C_i of the keyword of the content description information in the ith voice information segment.

For the method for determining the keyword in the ith voice information segment, reference may be made to the embodiment shown in FIG. 3 . Specifically, reference may be made to the method for determining the keyword of the first voice information segment in the embodiment shown in FIG. 3 , which is not repeated here. The method for determining the keyword of the content description information can be found in the embodiment shown in FIG. 3 , and it is unnecessary to describe it here. The implementation of the video segmentation device for determining the keyword in the i-th speech information segment and the word frequency vector of the keyword in the content description information in the i-th speech information segment can refer to the embodiment shown in FIG. 3 to determine the first The method for determining the keyword vector does not need to be repeated here.

604. The video segmentation apparatus determines a word frequency vector C_i of the keyword in the i-th speech information segment in the i-th speech information segment. The method of determining the word frequency vector of the keyword in the i-th speech information segment in the i-th speech information segment is the same as that of the keyword in the i-th speech information segment and the keyword of the content description information in the i-th speech information segment. The word frequency vector C_i in is determined in a similar manner, so it is unnecessary to describe it here.

After performing step 603 or step 604, the video segmentation apparatus may perform step 605 and step 606 in sequence.

605, the video segmentation apparatus determines the distance between C_i and C_(i-1). C_(i-1) is the key word of the i-1 th voice information segment determined by the video segmentation device (or the keyword of the i-1 th voice information segment and the keyword of the content description information) in the i- th Word frequency vector in 1 speech information segment. The i-1 th voice information fragment is a voice information fragment before the ith voice information fragment

606. If the distance between C_i and C_(i-1) is greater than the preset distance, it may be determined that the segmentation point is located before and after the i-th speech information segment. In the case that the video segmentation device determines that the segmentation point is located before and after the ith voice information segment, the segmentation point can be determined according to the pause point. The specific implementation manner of the video segmentation apparatus determining the segmentation point according to the pause point may refer to the embodiment shown in FIG. 3 , and it is unnecessary to describe it here.

If the distance between C_i and C_(i-1) is less than or equal to the preset distance, it can be considered that the segmentation point is not in the i-th speech information segment and the i-1-th speech information segment. In this case, the word frequency vector of the next piece of speech information and the word frequency vector of the i-th piece of speech information can continue to be determined.

FIG. 7 is a structural block diagram of a video segmentation apparatus provided according to an embodiment of the present application. As shown in FIG. 7 , the video segmentation apparatus 700 includes an acquisition unit 701 and a processing unit 702 .

The obtaining unit 701 is configured to obtain text information of the video to be processed and voice information of the video to be processed, wherein the text information includes at least one of a presentation in the video to be processed and content description information of the video to be processed.

The processing unit 702 is configured to determine the segment point of the video to be processed according to the text information and the voice information.

The processing unit 702 is further configured to segment the video to be processed according to the segment point.

For the specific functions and beneficial effects of the acquiring unit 701 and the processing unit 702, reference may be made to the methods shown in FIG. 3 to FIG. 6 , which will not be repeated here.

FIG. 8 is a structural block diagram of a video segmentation apparatus provided according to an embodiment of the present application. The video segmentation apparatus 800 shown in FIG. 8 includes: a processor 801 , a memory 802 and a transceiver 803 .

The processor 801, the memory 802 and the transceiver 803 communicate with each other through an internal connection path to transmit control and/or data signals.

The methods disclosed in the above embodiments of the present application may be applied to the processor 801 or implemented by the processor 801 . The processor 801 may be an integrated circuit chip with signal processing capability. In the implementation process, each step of the above-mentioned method may be completed by an integrated logic circuit of hardware in the processor 801 or an instruction in the form of software. The above-mentioned processor 801 may be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), an off-the-shelf programmable gate array (fieldprogrammable gate array, FPGA), or any other available processor. Programming logic devices, discrete gate or transistor logic devices, discrete hardware components. The methods, steps, and logic block diagrams disclosed in the embodiments of this application can be implemented or executed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in conjunction with the embodiments of the present application may be directly embodied as executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. Software modules can be located in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory or electrically erasable programmable memory, registers, etc. in the storage medium. The storage medium is located in the memory 802, and the processor 801 reads the instructions in the memory 802, and completes the steps of the above method in combination with its hardware.

Optionally, in some embodiments, the memory 802 may store instructions for performing the method performed by the video segmentation apparatus in the methods shown in FIGS. 3 to 6 . The processor 801 can execute the instructions stored in the memory 802 in combination with other hardware (such as the transceiver 803) to complete the steps of the video segmentation device in the method shown in FIG. 3 to FIG. 6 as described in the embodiment shown.

Embodiments of the present application further provide a chip, where the chip includes a transceiver unit and a processing unit. The transceiver unit may be an input/output circuit or a communication interface; the processing unit may be a processor, a microprocessor or an integrated circuit integrated on the chip. The chip can execute the method of the video segmentation apparatus in the above method embodiments.

Embodiments of the present application further provide a computer-readable storage medium, on which instructions are stored, and when the instructions are executed, the methods of the video segmentation apparatus in the above method embodiments are executed.

The embodiments of the present application further provide a computer program product including instructions, when the instructions are executed, the methods of the video segmentation apparatus in the above method embodiments are executed.

Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in this application can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of this application.

Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described systems, devices and units may refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

In the several embodiments provided in this application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.

The functions, if implemented in the form of software functional units and sold or used as independent products, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be embodied in the form of a software product in essence, or the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program codes .

The above are only specific embodiments of the present application, but the protection scope of the present application is not limited to this. should be covered within the scope of protection of this application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims (24)

1. A method of video segmentation, comprising:

the video segmentation device acquires text information of a video to be processed and voice information of the video to be processed, wherein the text information comprises at least one of a presentation in the video to be processed and content description information of the video to be processed, and the content description information is information for describing speaking content;

the video segmentation device determines segmentation points of the video to be processed according to the text information and the voice information;

and the video segmenting device segments the video to be processed according to the segmenting points.

2. The method of claim 1, wherein in the case that the text information includes the presentation, the video segmentation apparatus determining a segmentation point of the video to be processed according to the text information and the voice information comprises:

determining a switching point of the presentation, wherein the contents of the presentation before and after the switching point are different;

determining at least one stop point according to the voice information;

determining the segment point according to the switching point and the at least one pause point.

3. The method of claim 2, wherein the determining the segmentation point from the switching point and the at least one suspension point comprises:

determining the switching point as the segment point if it is determined that the switching point is the same as one of the at least one pause point;

and under the condition that any stopping point of the at least one stopping point is determined to be different from the switching point, determining one stopping point which is closest to the switching point in the at least one stopping point as the segmentation point.

4. The method of claim 2 or 3, wherein the determining a switch point for the presentation comprises: and determining the moment when the switching signal for instructing the switching of the content of the presentation is acquired as the switching point.

5. The method according to claim 2 or 3, wherein the text information further includes the content description information, and before the video segmentation apparatus determines the segmentation point of the video to be processed according to the text information and the speech information, the method further comprises:

and determining that the presentation time length of the current page of the presentation is less than or equal to a first preset time length and greater than a second preset time length.

6. The method of claim 1, wherein in the case that the text information includes the content description information, the video segmentation apparatus determining a segmentation point of the video to be processed according to the text information and the speech information comprises:

and determining the segmentation point of the video to be processed according to the voice information, the key word of the content description information and the stop point in the voice information.

7. The method of claim 6, wherein the voice information includes a first voice information segment and a second voice information segment, wherein the second voice information segment is a voice information segment preceding and adjacent to the first voice information segment,

the determining the segmentation point of the video to be processed according to the voice information, the keyword of the content description information and the stop point in the voice information comprises:

and determining a first segmentation point according to the first voice information segment, the second voice information segment, the keyword of the content description information and the pause point in the voice information, wherein the segmentation point of the video to be processed comprises the first segmentation point.

8. The method of claim 7, wherein determining a first segmentation point based on the first segment of speech information, the second segment of speech information, the keyword of the content description information, and a pause point in the speech information comprises:

determining the similarity of the first voice information segment and the second voice information segment according to the keywords of the first voice information segment, the keywords of the second voice information segment, the content of the first voice information segment, the content of the second voice information segment and the keywords of the content description information;

determining that the similarity of the first voice information segment and the second voice information segment is smaller than a similarity threshold;

and determining the first segmentation point according to the pause point in the voice information.

9. The method of claim 8, wherein the pause point in the speech information comprises a pause point within the first segment of speech information or a pause point adjacent to the first segment of speech information, and wherein determining the first segmentation point based on the pause point in the speech information comprises:

and determining the first segmentation point according to at least one of the number of pause points in the first voice information segment, the number of pause points adjacent to the first voice information segment, pause duration and words adjacent to the pause points.

10. The method according to any of claims 6 to 9, wherein the text information further comprises the presentation, and before the video segmentation means determines the segmentation points of the video to be processed from the text information and the speech information, the method further comprises:

determining that the presentation time length of the current page of the presentation is greater than a first preset time length; or alternatively

And determining that the presentation time length of the current page of the presentation is less than or equal to a second preset time length.

11. The method of any of claims 1-3, 6-9, further comprising: the video segmentation apparatus determines the digest of the segment according to the content of the segmented voice information, the keywords of the segmented voice information, and the keywords of a target text, wherein the target text includes at least one of the presentation and the content description information.

12. The method according to any one of claims 1 to 3 and 6 to 9, wherein the video to be processed is a real-time video stream, and the voice information of the video to be processed is the voice information of the real-time video stream from a start time or a last segmentation point of the real-time video stream to a current time.

13. A video segmentation apparatus, comprising:

the device comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring text information of a video to be processed and voice information of the video to be processed, the text information comprises at least one of a presentation in the video to be processed and content description information of the video to be processed, and the content description information is information used for describing speaking content;

the processing unit is used for determining the segmentation points of the video to be processed according to the text information and the voice information;

and the processing unit is further used for segmenting the video to be processed according to the segmentation points.

14. The video segmentation apparatus as set forth in claim 13, wherein the processing unit is specifically configured to determine a switching point of the presentation based on the text information and the speech information if the text information includes the presentation, the presentation being presented with different contents before and after the switching point;

determining at least one stop point according to the voice information;

determining the segmentation point according to the switching point and the at least one pause point.

15. Video segmentation unit as claimed in claim 14, characterized in that the processing unit is specifically adapted to

Determining the switching point as the segment point if it is determined that the switching point is the same as one of the at least one pause point;

and in the case that any one of the at least one pause point is determined to be different from the switching point, determining one of the at least one pause point closest to the switching point as the segmentation point.

16. The video segmenting device according to claim 14 or 15, characterized in that the processing unit is specifically configured to determine a time when a switch signal instructing to switch the content of the presentation is acquired as the switch point.

17. The video segmentation apparatus according to claim 14 or 15, wherein the processing unit is further configured to determine that a presentation time length of a current page of the presentation is less than or equal to a first preset time length and greater than a second preset time length before determining a segmentation point of the video to be processed according to the text information and the speech information in a case that the text information further includes the content description information.

18. The video segmentation apparatus as claimed in claim 13, wherein the processing unit is specifically configured to determine the segmentation point of the video to be processed according to the speech information, the keyword of the content description information, and a pause point in the speech information, if the text information includes the content description information.

19. The video segmentation apparatus of claim 18 wherein the voice information includes a first segment of voice information and a second segment of voice information, wherein the second segment of voice information is a segment of voice information preceding and adjacent to the first segment of voice information,

the processing unit is specifically configured to determine a first segmentation point according to the first speech information segment, the second speech information segment, the keyword of the content description information, and a pause point in the speech information, where the segmentation point of the video to be processed includes the first segmentation point.

20. The video segmentation apparatus as set forth in claim 19, wherein the processing unit is specifically configured to determine the similarity between the first voice message segment and the second voice message segment according to the keyword of the first voice message segment, the keyword of the second voice message segment, the content of the first voice message segment, the content of the second voice message segment, and the keyword of the content description information;

determining that the similarity of the first voice information segment and the second voice information segment is smaller than a similarity threshold;

and determining the first segmentation point according to the pause point in the voice information.

21. The video segmentation apparatus as set forth in claim 20, wherein the pause point in the speech information comprises a pause point in the first segment of speech information or a pause point adjacent to the first segment of speech information, and wherein the processing unit is specifically configured to determine the first segmentation point based on at least one of a number of pause points in the first segment of speech information, a number of pause points adjacent to the first segment of speech information, a pause duration, and a word adjacent to the pause point.

22. The video segmentation apparatus according to any one of claims 18 to 21, wherein the processing unit is further configured to determine that a presentation duration of a current page of the presentation is greater than a first preset duration before determining a segmentation point of the video to be processed according to the text information and the voice information in a case that the text information further includes the presentation; or

And determining that the presentation time length of the current page of the presentation is less than or equal to a second preset time length.

23. The video segmentation apparatus as set forth in any one of claims 13 to 15 and 18 to 21, wherein the processing unit is further configured to determine the abstract of the segment according to a content of the segmented voice information, a keyword of the segmented voice information, and a keyword of a target text, wherein the target text comprises at least one of the presentation and the content description information.

24. The video segmentation apparatus as claimed in any one of claims 13 to 15 and 18 to 21, wherein the video to be processed is a real-time video stream, and the voice information of the video to be processed is the voice information of the real-time video stream from a start time or a last segmentation point of the real-time video stream to a current time.

Images