CN115720278B - Synchronous processing method of sound and picture and related device - Google Patents
Synchronous processing method of sound and picture and related device Download PDFInfo
- Publication number
- CN115720278B CN115720278B CN202211368580.0A CN202211368580A CN115720278B CN 115720278 B CN115720278 B CN 115720278B CN 202211368580 A CN202211368580 A CN 202211368580A CN 115720278 B CN115720278 B CN 115720278B
- Authority
- CN
- China
- Prior art keywords
- picture
- video
- instruction
- buffer
- intelligent
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The application is suitable for the technical field of intelligent video and audio equipment, provides a synchronous processing method and a related device of sound and pictures applied to intelligent video and audio equipment, and aims to solve the problem that the intelligent video and audio equipment in the prior art is asynchronous when finally outputting pictures and sound. The method mainly comprises the steps of increasing the buffer memory number of picture frames of a playing picture of the intelligent video equipment in a buffer memory according to a picture delay instruction when the picture delay instruction is received, and reducing the buffer memory number of the picture frames of the playing picture of the intelligent video equipment in the buffer memory according to the picture advance instruction when the picture advance instruction is received.
Description
Technical Field
The application belongs to the technical field of intelligent video and audio equipment, and particularly relates to a synchronous processing method and a related device for sound and pictures applied to intelligent video and audio equipment.
Background
The media resources played by the intelligent television comprise sound and picture resources on the same track in time sequence, the intelligent television needs to acquire the media resources first for realizing sound and picture output in the media resources, then the media resources are subjected to related protocol analysis, audio/video decoding, synchronous rendering and other works, and finally the picture is output through the screen of the intelligent television and the sound is output through the loudspeaker or an external sound of the intelligent television. In the above process, from the audio/video decoding, the video and the audio are processed in different paths, each link after the video and the audio are processed separately may cause a certain time difference between the video and the audio, and finally when the screen of the smart television outputs the picture and the loudspeaker of the smart television or the external sound is output, the user may feel that the picture and the sound are not synchronous, and the watching experience is affected.
In the prior art, a solution for realizing synchronization of sound and pictures is not found to be arranged for a user at the intelligent television end.
Disclosure of Invention
The application aims to provide a synchronous processing method and a related device for sound and images, and aims to solve the problem that intelligent video and audio equipment in the prior art is not synchronous when finally outputting images and sound.
In a first aspect, a method for synchronously processing a picture and a sound is provided, and the method is applied to an intelligent video and audio device, and includes:
when receiving a picture delay instruction, increasing the buffer quantity of picture frames of a picture played by the intelligent video and audio equipment in a buffer according to the picture delay instruction;
when receiving the picture advance command, the picture advance command is used for reducing the buffer number of picture frames of the playing picture of the intelligent video and audio equipment in the buffer according to the picture advance command.
Optionally, the method further comprises:
receiving the synchronous processing instruction, wherein the synchronous processing instruction is used for starting and adjusting the output coordination of the picture and the sound at the intelligent video and audio equipment;
and activating the buffer memory in a locking state to adjust the buffer memory quantity of the picture frames of the intelligent video and audio equipment playing picture according to the synchronous processing instruction.
Optionally, the method further comprises:
receiving a synchronous processing completion instruction;
And locking the buffer quantity of the picture frames of the current intelligent video and audio equipment playing picture buffered by the buffer according to the synchronous processing completion instruction.
Optionally, after activating the function of adjusting the buffer number of the frame of the playback frame of the intelligent audio/video device by the buffer in the locked state according to the synchronization processing instruction, the method further includes:
Receiving current sound emitted by the intelligent video and audio equipment when playing video through a microphone;
Judging whether the current sound is aligned in time with a standard sound corresponding to a picture when the intelligent video and audio equipment plays video;
When the current sound is delayed than the standard sound corresponding to the picture when the intelligent video and audio equipment plays the video, generating a picture delay instruction;
When the current sound is aligned with the standard sound corresponding to the picture when the intelligent video and audio equipment plays the video, generating the synchronous processing completion instruction;
And when the current sound is earlier than the standard sound corresponding to the picture when the intelligent video and audio equipment plays the video, generating the picture advance instruction.
Optionally, the receiving, by the microphone, the current sound made when the intelligent audio/video device plays the video includes:
Receiving current sound emitted by a loudspeaker of the intelligent video and audio equipment when the intelligent video and audio equipment plays video through a microphone of the intelligent video and audio equipment, or,
And receiving the current sound emitted by the sound box which is wirelessly connected when the intelligent audio-visual equipment plays the video through the microphone of the intelligent audio-visual equipment.
Optionally, the intelligent audio-visual device is one of an intelligent television, an intelligent projector, an intelligent mobile phone, an intelligent tablet, a notebook computer and a desktop computer.
Optionally, the receiving the synchronization processing instruction includes:
The synchronous processing instruction is generated by triggering a first physical key of the intelligent video-audio equipment, or,
Triggering a first virtual key of the intelligent video and audio equipment to generate the synchronous processing instruction;
the receiving a synchronization processing completion instruction includes:
The synchronous processing completion instruction is generated by triggering a second physical key of the intelligent video and audio equipment, or,
And triggering a second virtual key of the intelligent video and audio equipment to generate the synchronous processing completion instruction.
In a second aspect, the present application provides a synchronous processing system for images and sounds, which is applied to intelligent audio-visual equipment, and includes:
the adding unit is used for adding the buffer quantity of the picture frames of the intelligent video and audio equipment playing picture in the buffer according to the picture delay instruction when the picture delay instruction is received;
and the reducing unit is used for reducing the buffer quantity of the picture frames of the playing picture of the intelligent video equipment in the buffer according to the picture advance instruction when the picture advance instruction is received.
Optionally, the system further comprises:
The receiving unit is used for receiving the synchronous processing instruction, and the synchronous processing instruction is used for starting and adjusting the output coordination of the picture and the sound at the intelligent video and audio equipment;
and the activating unit is used for activating the function of adjusting the buffer quantity of the buffer in the locked state for the picture frames of the intelligent video and audio equipment playing picture according to the synchronous processing instruction.
Optionally, the system further comprises:
The receiving unit is also used for receiving a synchronous processing completion instruction;
And the locking unit is used for locking the buffer quantity of the picture frames of the current intelligent video and audio equipment playing picture buffered by the buffer according to the synchronous processing completion instruction.
Optionally, the system further comprises:
the receiving unit is also used for receiving the current sound emitted by the intelligent video and audio equipment when playing the video through the microphone;
The judging unit is used for judging whether the current sound is aligned in time with the standard sound corresponding to the picture when the intelligent video and audio equipment plays the video;
The generating unit is used for generating the picture delay instruction when the current sound is delayed than the standard sound corresponding to the picture when the intelligent video and audio equipment plays the video;
the generating unit is further configured to generate the synchronization processing completion instruction when the current sound is aligned in time with a standard sound corresponding to a picture when the intelligent audio/video device plays a video;
The generating unit is further configured to generate the picture advance instruction when the current sound is advanced compared with a standard sound corresponding to a picture when the intelligent video and audio device plays the video.
Optionally, the receiving unit is specifically configured to, when receiving, through a microphone, a current sound made when the intelligent audio/video device plays a video:
Receiving current sound emitted by a loudspeaker of the intelligent video and audio equipment when the intelligent video and audio equipment plays video through a microphone of the intelligent video and audio equipment, or,
And receiving the current sound emitted by the sound box which is wirelessly connected when the intelligent audio-visual equipment plays the video through the microphone of the intelligent audio-visual equipment.
Optionally, the intelligent audio-visual device is one of an intelligent television, an intelligent projector, an intelligent mobile phone, an intelligent tablet, a notebook computer and a desktop computer.
Optionally, when the receiving unit receives the synchronization processing instruction, the receiving unit is specifically configured to:
The synchronous processing instruction is generated by triggering a first physical key of the intelligent video-audio equipment, or,
Triggering a first virtual key of the intelligent video and audio equipment to generate the synchronous processing instruction;
When the receiving unit receives the synchronization processing completion instruction, the receiving unit is specifically configured to:
The synchronous processing completion instruction is generated by triggering a second physical key of the intelligent video and audio equipment, or,
And triggering a second virtual key of the intelligent video and audio equipment to generate the synchronous processing completion instruction.
In a third aspect, the present application provides a computer device comprising:
Processor, memory, bus, input/output interface, network interface;
the processor is connected with the memory, the input/output interface and the network interface through the bus;
the memory stores a program;
The processor, when executing the program stored in the memory, implements the method for processing synchronization of a picture and a sound according to any one of the foregoing first aspects.
In a fourth aspect, the present application provides a computer storage medium having stored therein instructions which, when executed on a computer, cause the computer to perform the method for processing images and sounds according to any one of the preceding first aspects.
In a fifth aspect, the present application provides a computer program product which, when executed on a computer, causes the computer to perform a method of synchronous processing of pictures and sounds as set forth in any one of the preceding first aspects.
The above technical solution can be seen that the embodiment of the application has the following advantages:
The application discloses a synchronous processing method of pictures and sounds, which is applied to intelligent video equipment, wherein the intelligent video equipment is provided with a buffer for buffering the picture frames of a playing picture in a certain quantity, when a picture delay instruction is received, the buffer quantity of the buffered picture frames of the playing picture of the intelligent video equipment is increased in the buffer according to the picture delay instruction, the re-delay adjustment of the playing picture of the intelligent video equipment on the premise that the playing sound is unchanged is realized, when a picture advance instruction is received, the buffer quantity of the buffered picture frames of the playing picture of the intelligent video equipment is reduced in the buffer according to the picture advance instruction, the reduced delay adjustment of the playing picture of the intelligent video equipment on the premise that the playing sound is unchanged is realized, and because the prior art is that the sound output of the intelligent video equipment is delayed relative to the picture output when the content of media resources is played, the implementation can realize the control of different delay time lengths of the picture by controlling the buffer quantity of the picture frames of the playing picture in the buffer, and the sound of the intelligent video equipment is adapted to the playing picture on the premise that the different delay time lengths of the playing picture is controlled, so that the picture output of the intelligent video equipment is synchronous with the sound.
Drawings
FIG. 1 is a flowchart of a method for synchronizing a frame and a sound according to an embodiment of the present application;
FIG. 2 is a flowchart illustrating a method for synchronizing a frame and a sound according to another embodiment of the present application;
FIG. 3 is a schematic diagram illustrating an embodiment of a synchronous processing system for picture and sound according to the present application;
FIG. 4 is a schematic diagram illustrating an embodiment of a synchronous processing system for picture and sound according to the present application;
FIG. 5 is a schematic diagram illustrating the structure of a computer device according to an embodiment of the present application;
FIG. 6 is a schematic structural connection diagram of an embodiment of the intelligent audio/video device and the sound box of the present application;
fig. 7 is a schematic diagram of a data flow of an embodiment of the prior art when an intelligent audio/video device plays a media asset.
Detailed Description
The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.
It should be noted that, the intelligent audio-visual device in this embodiment refers to a device for playing media resources, and the media resources in this embodiment refer to electronic resources with time co-track sounds and pictures, such as videos of tv dramas, movies, and the like. The intelligent video and audio equipment capable of playing the media resources can be one of an intelligent television, an intelligent projector, an intelligent mobile phone, an intelligent tablet, a notebook computer, a desktop computer and the like.
Referring to fig. 7, when the smart audio/video device is taken as an example of the smart audio/video device, in order to realize the respective output of the sound and the picture in the media resource, the smart television needs to obtain the media resource 800 first, then obtain the picture frame sequence set 810 based on the sequence of time and the sound frame sequence set 820 based on the sequence of time after the media resource 800 is parsed, decoded and rendered by the relevant protocol, finally display the picture frame 811 in the picture frame sequence set 810 according to the sequence of time through the screen of the smart television, and play the sound frame 821 in the sound frame sequence set 820 according to the sequence of time through the loudspeaker of the smart television, and when the smart television is connected with the sound box with the same bluetooth function in a pairing manner, the sound frame 821 can be output by the sound box, specifically, the smart television converts the sound frame sequence set 820 into the sound frame 821 "ready to be played through bluetooth and transmits the sound frame 821 to the sound box through bluetooth, and obtains the sound frame 821 ready to be played after the corresponding processing is received, and then plays the sound frame 821. In the above process, from the start of audio/video decoding, the video frames and the video sounds are processed in different paths, each link after the video frames and the video sounds are processed separately may cause a certain time difference between the video frames and the video sounds, and finally when the video frames are output through the screen of the smart television and the video sounds are output through the loudspeaker of the smart television or the external sound equipment, the user may feel that the frames and the sounds are not synchronous, and the watching experience is affected. Particularly, when the smart television is wirelessly connected with an external sound box with a Bluetooth function through Bluetooth, the smart television needs to compress a sound signal and then send the compressed sound to the sound box through Bluetooth, and the sound box decompresses the compressed sound and then plays the sound, so that more time is consumed compared with the playing of the sound through a loudspeaker of the smart television, a user can feel that the playing picture of the smart television is obviously faster than the sound heard from the sound box, and the watching experience of the user is influenced.
In order to solve the problem that the intelligent audio-visual equipment is asynchronous when finally outputting pictures and sounds in the prior art, particularly when the intelligent audio-visual equipment plays the sounds of media resources through an external sound box, the sound played by the sound box is asynchronous with the display pictures of the intelligent audio-visual equipment in time sequence. Referring to fig. 6, the intelligent audio/video device of the present embodiment has a buffer added to the conventional intelligent audio/video device to solve the above-mentioned technical problems. The method for realizing the delay of the intelligent video and audio equipment in the final output picture can be completed in a mode of realizing the frame buffer (frame buffer) of the picture frame by adopting a buffer. The set of picture frame sequences 810 consists of a frame-by-frame still picture (picture frame 811), each of which needs to be frame buffered before being displayed. The display module displays the frames in the buffer frame by frame, so that the dynamic frames can be seen. When the user hopes that the display module realizes the picture delay, the effect of adjusting the display picture delay can be achieved by adjusting the buffer quantity of the picture frames in the buffer. Specifically, the intelligent audio/video device 600 mainly includes a processor 601, a speaker 602, a buffer 603, a display module 604, a communication module 605, a key module 606, and a power module (not shown). The processor 601 is mainly used for performing related protocol analysis, audio/video decoding, rendering and other operations on the media resource 800, so as to obtain a frame sequence set 810 based on a time sequence and having a sequence, The processor 601 sends the picture frames 811 in the picture frame sequence set 810 to the buffer 603 according to the time sequence, when the communication module 605 determines that no external sound box is provided, the processor 601 also transmits the sound frames 821 in the sound frame sequence set 820 to the loudspeaker 602 according to the time sequence, and when the communication module 605 determines that the external sound box 700 is provided, the processor 601 also transmits the sound frames 821 in the sound frame sequence set 820 to the external sound box 700 according to the time sequence through the communication module 605. The loudspeaker 602 is mainly used for playing the sound frame 821 transmitted by the processor 601 to realize sound output. The buffer 603 is used for buffering a plurality of frames according to the control logic adjusted by the processor 601, and then transmitting the frames to the display module 604 according to the first-in first-out sequence. The display module 604 mainly enables the user to display the picture frames transmitted by the buffer 603. The communication module 605 is mainly configured to determine whether to establish a connection with an external speaker box, if a connection (e.g. bluetooth connection) is successfully established with the speaker box 700, feedback the external speaker box 700 to the processor 601, so that the processor 601 gives up to play the sound frame 821 using a speaker carried by itself, and preferentially select the external speaker box 700 to play the sound frame 821, and if no connection (e.g. bluetooth connection) is established with the speaker box 700, feedback the speaker box 700 without external connection to the processor 601, so that the processor 601 uses the speaker carried by itself to play the sound frame 821. The key module 606 is mainly used for receiving an operation instruction of the user on the audio/video enabled device 600, and transmitting the related operation instruction to the processor 601 for execution. The power module is mainly used for providing proper working voltages for the processor 601, the loudspeaker 602, the buffer 603, the display module 604, the communication module 605, the key module 606 and the like.
Research shows that the intelligent audio-visual equipment is asynchronous when finally outputting pictures and sounds, particularly when the intelligent audio-visual equipment plays the sounds of media resources through an external sound box, the sound played by the sound box is asynchronous with the display pictures of the intelligent audio-visual equipment in time sequence, and the intelligent audio-visual equipment mainly shows that the pictures played by the intelligent audio-visual equipment are faster than the sounds. In order to solve the technical problem that the picture played by the intelligent video and audio equipment is faster than the sound, two solutions are generally adopted, namely, the first solution is that the picture playing needs to be regulated to be delayed on the premise of keeping the sound continuously played according to the time sequence order, so that the user feels that the picture and the sound are synchronous in output, and the second solution is that the sound playing needs to be regulated to be advanced on the premise of keeping the picture continuously played according to the time sequence order, so that the user also feels that the picture and the sound are synchronous in output. However, in reality, the second scheme has relatively limited effect, and especially when the intelligent audio-visual device plays the sound of the media resource through the external sound box, the intelligent audio-visual device needs to consume a long time to play the sound sent to the external sound box, and the time length for adjusting the sound to play is not long enough, which is usually insufficient to make up the time difference of the asynchronization between the picture and the sound. The first solution described above is mainly adopted in this embodiment.
Therefore, based on the above understanding of the intelligent audio/video device 600 of the present application, please refer to fig. 1, and the process of solving the above technical problem is described in the following embodiment, the present application is applied to an embodiment of a method for synchronously processing a picture and a sound of an intelligent audio/video device, which includes:
101. when receiving the picture delay instruction, increasing the buffer quantity of picture frames of the intelligent video and audio equipment playing picture in the buffer according to the picture delay instruction.
When the user watches the video played by the intelligent audio-visual device and feels that the picture is faster than the sound, the step can receive the picture delay instruction triggered by the user through the key module 606, and then increase the buffer number of the picture frames of the video played by the intelligent audio-visual device in the buffer 603 according to the picture delay instruction, so as to realize the re-delay adjustment of the played picture of the intelligent audio-visual device on the premise that the played sound is unchanged. For example, each time the user triggers a picture delay command (the picture delay command issued by the user from the physical key is received by the key module 606), this step stores a picture frame of multiple X frames in the buffer 603, where X is a positive integer greater than or equal to 1. In general, the playback frame of the intelligent video and audio device is 60Hz, that is, 60 frames per second, and the buffer 603 stores 1 frame of the frame to delay the frame of 1/60 second, and 6 frames of the buffer may delay the frame of 1/10 second (100 ms). The sound box externally connected in the bluetooth mode is usually delayed for 150ms, and the synchronization of the output picture of the intelligent audio-visual device and the output sound of the sound box can be realized by storing 9 picture frames in the buffer 603. 102. When receiving the picture advance command, the buffer memory is used for reducing the buffer memory number of the picture frames of the playing picture of the intelligent video and audio equipment according to the picture advance command.
When the user stores too many frames in the buffer 603 in the operation of step 101, the situation that the video is slower than the sound may be perceived when the user watches the intelligent audio/video device, then the step may receive the user triggered frame advance command through the key module 606, and then reduce the buffer number of frames of the playing frame of the intelligent audio/video device in the buffer 603 according to the frame advance command, so as to realize the relative advance adjustment of the playing frame of the intelligent audio/video device on the premise that the playing sound is unchanged. For example, each time the user triggers a picture advance command (the picture advance command issued by the user from the physical key is received by the key module 606), this step releases the picture frames of multiple X frames from the buffer 603, where X is a positive integer greater than or equal to 1, until the buffer 603 has no picture frames buffered, and the limit of picture advance adjustment is reached. In general, the playing frame of the intelligent video and audio device is 60Hz, that is, 60 frames per second, the buffer 603 releases 1 frame of the frame to bring about 1/60 second of the frame advance, and 6 frames of the buffer bring about 1/10 second (100 ms) of the advance.
As can be seen, in the prior art, since the sound output of the intelligent audio-visual device when playing the media resource content is delayed relative to the picture output, the implementation can realize the control of different delay time durations of the picture by controlling the buffer number of the picture frames of the playing picture in the buffer, and adapt the sound delayed than the playing picture in the intelligent audio-visual device by controlling the different delay time durations of the playing picture, so as to realize the synchronization of the picture output by the intelligent audio-visual device and the sound.
Referring to fig. 2, another embodiment of a method for processing images and sounds synchronously applied to an intelligent video and audio device of the present application includes:
201. And receiving a synchronous processing instruction, wherein the synchronous processing instruction is used for starting the output cooperation of the intelligent video and audio equipment for adjusting the picture and the sound.
It can be understood that, because the synchronous calibration work of the picture and the sound of the intelligent audio-visual device is a matter which has a certain technical requirement and can directly influence the use experience of the user, the step needs to definitely receive the synchronous processing instruction sent by the user to the intelligent audio-visual device to activate the related program in order to realize the synchronous picture and the sound of the user layer by starting the output cooperation of the picture and the sound adjusting at the intelligent audio-visual device due to the safety. For example, the step of receiving the synchronization instruction triggered by the user may be that the synchronization instruction is generated by triggering a first physical key of the intelligent audio-visual device, or that the synchronization instruction is generated by triggering a first virtual key (a virtual key on the touch screen) of the intelligent audio-visual device. The form of receipt of the synchronization processing instruction is not limited herein.
202. Activating the buffer in a locking state to adjust the buffer quantity of the picture frames of the playing picture of the intelligent video equipment according to the synchronous processing instruction.
For safety reasons, this step needs to activate the function of adjusting the buffer memory 603 in a locked state to buffer the number of frames of the playing frame of the intelligent audio/video device according to the synchronization processing instruction when the synchronization processing instruction sent by the user to the intelligent audio/video device is explicitly received in step 201, thereby allowing the buffer memory 603 to store the number of frames of the frame in the buffer memory 603 in the subsequent steps to increase or decrease.
203. The current sound emitted when the intelligent video and audio equipment plays the video is received through the microphone.
In some embodiments, the function of "automatic synchronization between picture and sound" of the intelligent audio-visual device may be further implemented, when the intelligent audio-visual device has the microphone recording function, after the step 202 activates the function of adjusting the buffer number of the buffer in the locked state for the picture frame of the playing picture of the intelligent audio-visual device, the step may receive the current sound (the sound box sent by the loudspeaker of the intelligent audio-visual device or the sound box sent by the wireless connection) sent by the intelligent audio-visual device when the intelligent audio-visual device plays the video.
204. Whether the current sound is aligned with the standard sound corresponding to the picture when the intelligent video and audio equipment plays the video in time is judged, if the current sound is delayed than the standard sound corresponding to the picture when the intelligent video and audio equipment plays the video, step 205 is executed, if the current sound is advanced than the standard sound corresponding to the picture when the intelligent video and audio equipment plays the video, step 207 is executed, and if the current sound is aligned with the standard sound corresponding to the picture when the intelligent video and audio equipment plays the video in time, step 209 is executed.
It should be noted that when the intelligent av device decodes the audio and video of the media resource 800, the corresponding relation between each frame of picture and the corresponding standard sound in the time sequence order can be known, after the current sound (the speaker of the intelligent av device itself plays or the external sound box plays) sent when the intelligent av device plays the video is known in step 203, the step can determine whether the current sound is aligned in time with the standard sound corresponding to the picture when the intelligent av device plays the video, if the current sound is delayed compared with the standard sound corresponding to the picture when the intelligent av device plays the video, the time difference between the current sound and the standard sound corresponding to the picture can be calculated, and then how many frames of the picture are needed to be stored in the buffer, if the current sound is advanced compared with the standard sound corresponding to the picture when the intelligent av device plays the video, the time difference is further calculated, how many frames of the picture need to be released from the buffer, and if the current sound is aligned in time with the standard sound corresponding to the picture when the intelligent av device plays the video, the picture is synchronous with the current sound when the intelligent av device plays the video.
205. And generating a picture postponement instruction.
When it is determined in step 204 that the current sound is delayed than the standard sound corresponding to the picture when the intelligent video and audio device plays the video, the step actively generates or invokes a preset picture delay instruction, where the picture delay instruction is used to increase the number of the buffered picture frames in the buffer.
206. And increasing the buffer quantity of the picture frames of the picture played by the intelligent video and audio equipment in the buffer according to the picture delay instruction.
According to the picture delay instruction in step 205, the calculated number of frames of the picture to be stored in the buffer is carried along with the picture delay instruction, so that the buffer number of the corresponding number of frames of the picture to be played by the intelligent video and audio equipment is increased in the buffer, and the execution of step 204 is continuously triggered.
207. And generating a picture advance command.
After determining that the current sound is earlier than the standard sound corresponding to the picture when the intelligent video and audio device plays the video in step 204, the step actively generates or invokes a preset picture advance command, where the picture advance command is used to reduce the number of frames in the buffer.
208. And according to the picture advance instruction, the buffer memory is used for reducing the buffer memory number of picture frames of the playing picture of the intelligent video and audio equipment.
According to the picture advance command in step 207, the calculated buffer number of picture frames to be reduced from the buffer is carried with the picture advance command, so that the buffer number of picture frames of the picture played by the intelligent video and audio equipment is reduced by a corresponding number in the buffer, and the step 204 is triggered and executed continuously.
209. A synchronization process completion instruction is generated.
When it is determined in step 204 that the current sound is aligned with the standard sound corresponding to the picture when the intelligent audio/video device plays the video, it is proved that the picture is synchronous with the current sound when the intelligent audio/video device plays the video.
It should be noted that, in some embodiments, the synchronization processing completion instruction of this step may also be generated by triggering the second physical key of the smart audio/video device, or by triggering the virtual key on the second virtual key touch screen of the smart audio/video device).
210. And locking the buffer quantity of the picture frames of the playing picture of the intelligent video equipment buffered in the current buffer according to the synchronous processing completion instruction.
According to the synchronization processing completion instruction of step 209, the step locks the buffer number of the picture frames in the buffer for buffering the play picture of the intelligent video equipment, thereby completing the automatic synchronization of the picture and the sound of the intelligent video equipment.
The above embodiments describe a method for processing images and sounds of an intelligent audio/video device according to the present application, and the following describes a system for processing images and sounds of an intelligent audio/video device according to the present application, referring to fig. 3, an embodiment of the system for processing images and sounds of an intelligent audio/video device according to the present application includes:
an adding unit 301, configured to, when receiving a picture delay instruction, increase, in a buffer, the number of buffers for picture frames of a picture played by the intelligent audio/video device according to the picture delay instruction;
And the reducing unit 302 is configured to reduce, in the buffer, the number of buffered frames of the playback frame of the intelligent audio/video device according to the picture advance command when the picture advance command is received.
The operation performed by the synchronous processing system of the picture and the sound applied to the intelligent video and audio device in this embodiment is similar to the operation performed in the foregoing embodiment of fig. 1, and will not be described herein.
The application discloses a synchronous processing system of pictures and sounds, which is applied to intelligent video equipment, wherein the intelligent video equipment is provided with a buffer for buffering the picture frames of a playing picture in a certain quantity, when a picture delay instruction is received, the buffer quantity of the buffered picture frames of the playing picture of the intelligent video equipment is increased in the buffer according to the picture delay instruction, the re-delay adjustment of the playing picture of the intelligent video equipment on the premise that the playing sound is unchanged is realized, when a picture advance instruction is received, the buffer quantity of the buffered picture frames of the playing picture of the intelligent video equipment is reduced in the buffer according to the picture advance instruction, the reduction delay adjustment of the playing picture of the intelligent video equipment on the premise that the playing sound is unchanged is realized, and because the prior art is that the sound output of the intelligent video equipment is delayed relative to the picture output when the content of media resources is played, the implementation can realize the control of different delay time lengths of the picture by controlling the buffer quantity of the picture frames of the playing picture in the buffer, and the sound of the intelligent video equipment is adapted to the playing picture on the premise that the different delay time lengths of the playing picture is controlled, so that the picture output of the intelligent video equipment is synchronous with the sound.
Referring to fig. 4, another embodiment of a synchronous processing system for picture and sound applied to an intelligent video and audio device of the present application includes:
an adding unit 401, configured to, when receiving a picture delay instruction, increase the number of buffers for picture frames of a picture played by the intelligent video and audio device in a buffer according to the picture delay instruction;
And the reducing unit 402 is configured to reduce, in the buffer, the number of buffered frames of the playback frame of the intelligent audio/video device according to the picture advance command when the picture advance command is received.
Optionally, the system further comprises:
A receiving unit 403, configured to receive the synchronization processing instruction, where the synchronization processing instruction is used to start and adjust output coordination between the picture and the sound at the intelligent audio-visual device;
And the activating unit 404 is configured to activate, according to the synchronization processing instruction, a function of adjusting the buffer number of the buffer in the locked state for the frame of the playback frame of the intelligent audio/video device.
Optionally, the system further comprises:
the receiving unit 403 is further configured to receive a synchronization processing completion instruction;
and the locking unit 405 is configured to lock, according to the synchronization processing completion instruction, the number of buffers for buffering the current frame of the playback frame of the intelligent audio/video device in the buffer.
Optionally, the system further comprises:
the receiving unit 403 is further configured to receive, through a microphone, a current sound emitted when the intelligent audio/video device plays a video;
A judging unit 406, configured to judge whether the current sound is aligned in time with a standard sound corresponding to a picture when the intelligent audio/video device plays a video;
A generating unit 407, configured to generate the picture delay instruction when the current sound is delayed compared with a standard sound corresponding to a picture when the intelligent audio/video device plays a video;
The generating unit 407 is further configured to generate the synchronization processing completion instruction when the current sound is aligned in time with a standard sound corresponding to a picture when the intelligent audio/video device plays a video;
The generating unit 407 is further configured to generate the picture advance instruction when the current sound is advanced compared with a standard sound corresponding to a picture when the intelligent audio/video device plays a video.
Optionally, the receiving unit 403 is specifically configured to, when receiving, through a microphone, a current sound made when the smart audio/video device plays a video:
Receiving current sound emitted by a loudspeaker of the intelligent video and audio equipment when the intelligent video and audio equipment plays video through a microphone of the intelligent video and audio equipment, or,
And receiving the current sound emitted by the sound box which is wirelessly connected when the intelligent audio-visual equipment plays the video through the microphone of the intelligent audio-visual equipment.
Optionally, the intelligent audio-visual device is one of an intelligent television, an intelligent projector, an intelligent mobile phone, an intelligent tablet, a notebook computer and a desktop computer.
Optionally, when the receiving unit 403 receives the synchronization processing instruction, the method is specifically configured to:
The synchronous processing instruction is generated by triggering a first physical key of the intelligent video-audio equipment, or,
Triggering a first virtual key of the intelligent video and audio equipment to generate the synchronous processing instruction;
When the receiving unit 403 receives the synchronization processing completion instruction, the method specifically is used for:
The synchronous processing completion instruction is generated by triggering a second physical key of the intelligent video and audio equipment, or,
And triggering a second virtual key of the intelligent video and audio equipment to generate the synchronous processing completion instruction.
The operations executed by the synchronous processing system of the picture and the sound applied to the intelligent video and audio device in this embodiment are similar to those executed in the foregoing embodiment of fig. 1 or fig. 2, and will not be described in detail herein.
Referring to fig. 5, an embodiment of a computer device according to an embodiment of the present application includes:
The computer device 500 may include one or more processors (central processing units, CPU) 501 and memory 502, with one or more applications or data stored in the memory 502. Wherein the memory 502 is volatile storage or persistent storage. The program stored in memory 502 may include one or more modules, each of which may include a series of instruction operations in a computer device. Still further, the processor 501 may be configured to communicate with the memory 502 and execute a series of instruction operations in the memory 502 on the computer device 500. The computer device 500 may also include one or more network interfaces 503, one or more input/output interfaces 504, and/or one or more operating systems, such as Windows Server, mac OS, unix, linux, freeBSD, etc. The processor 501 may perform the operations performed in the embodiments shown in fig. 1 or fig. 2, and detailed descriptions thereof are omitted herein.
In the several embodiments provided in the embodiments of the present application, it should be understood by those skilled in the art that the disclosed systems, apparatuses and methods may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units. The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied in essence or a part contributing to the prior art or all or part of the technical solution in the form of a software product stored in a storage medium, including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of the embodiments of the present application. The storage medium includes a usb disk, a removable hard disk, a read-only memory (ROM), a random-access memory (RAM, random access memory), a magnetic disk, an optical disk, or other various media capable of storing program codes.
The foregoing description of the preferred embodiments of the application is not intended to be limiting, but rather is intended to cover all modifications, equivalents, or alternatives falling within the spirit and principles of the application.
Claims (7)
1. The synchronous processing method of the picture and the sound is characterized by being applied to intelligent video and audio equipment and comprising the following steps:
receiving a synchronous processing instruction, wherein the synchronous processing instruction is used for starting and adjusting the output coordination of a picture and sound in the intelligent video and audio equipment;
Activating a buffer in a locking state to adjust the buffer quantity of the picture frames of the intelligent video and audio equipment playing picture according to the synchronous processing instruction;
when receiving a picture delay instruction, increasing the buffer number of picture frames of a picture played by the intelligent video equipment in the buffer according to the picture delay instruction, wherein each time the picture delay instruction is received, the picture frames of X frames are stored in the buffer, and X is a positive integer greater than or equal to 1;
When receiving a picture advance instruction, according to the picture advance instruction, reducing the buffer number of picture frames of a playing picture of the intelligent video equipment in the buffer, and releasing the picture frames of X frames to the buffer when receiving the picture advance instruction once;
The method further comprises the steps of:
receiving a synchronous processing completion instruction;
locking the buffer quantity of the picture frames of the current intelligent video and audio equipment playing picture buffered by the buffer according to the synchronous processing completion instruction;
the receiving the synchronization processing instruction includes:
triggering a first physical key of the intelligent video and audio equipment to generate the synchronous processing instruction;
the receiving a synchronization processing completion instruction includes:
And triggering a second physical key of the intelligent video and audio equipment to generate the synchronous processing completion instruction.
2. The synchronization processing method according to claim 1, wherein after activating the function of adjusting the buffer number of the picture frames of the smart av device play picture by the buffer in the locked state according to the synchronization processing instruction, the method further comprises:
Receiving current sound emitted by the intelligent video and audio equipment when playing video through a microphone;
Judging whether the current sound is aligned in time with a standard sound corresponding to a picture when the intelligent video and audio equipment plays video;
When the current sound is delayed than the standard sound corresponding to the picture when the intelligent video and audio equipment plays the video, generating a picture delay instruction;
When the current sound is aligned with the standard sound corresponding to the picture when the intelligent video and audio equipment plays the video, generating the synchronous processing completion instruction;
And when the current sound is earlier than the standard sound corresponding to the picture when the intelligent video and audio equipment plays the video, generating the picture advance instruction.
3. The synchronization processing method according to claim 2, wherein the receiving, by a microphone, a current sound made when the smart audiovisual device plays a video includes:
Receiving current sound emitted by a loudspeaker of the intelligent video and audio equipment when the intelligent video and audio equipment plays video through a microphone of the intelligent video and audio equipment, or,
And receiving the current sound emitted by the sound box which is wirelessly connected when the intelligent audio-visual equipment plays the video through the microphone of the intelligent audio-visual equipment.
4. The synchronization processing method according to claim 2, wherein the smart audio/video device is one of a smart television, a smart projector, a smart phone, a smart tablet, a notebook computer, and a desktop computer.
5. Synchronous processing system of picture and sound, its characterized in that is applied to intelligent audio-visual equipment, includes:
the receiving unit is used for receiving a synchronous processing instruction, and the synchronous processing instruction is used for starting the adjustment picture to be matched with the output of the intelligent video and audio equipment;
the activating unit is used for activating the function of adjusting the buffer quantity of the buffer in the locking state to the picture frames of the intelligent video and audio equipment playing picture according to the synchronous processing instruction;
The intelligent video equipment comprises an increasing unit, a storage unit and a display unit, wherein the increasing unit is used for increasing the buffer number of picture frames of a picture played by the intelligent video equipment in the buffer according to the picture delay instruction when receiving the picture delay instruction, and storing the picture frames of X frames into the buffer when receiving the picture delay instruction once, wherein X is a positive integer greater than or equal to 1;
The reducing unit is used for reducing the buffer quantity of the picture frames of the playing picture of the intelligent video equipment according to the picture advance instruction in the buffer when the picture advance instruction is received, and releasing the picture frames of X frames to the buffer when the picture advance instruction is received once;
The system further comprises:
The receiving unit is also used for receiving a synchronous processing completion instruction;
The locking unit is used for locking the buffer quantity of the picture frames of the current intelligent video and audio equipment playing picture buffered by the buffer according to the synchronous processing completion instruction;
when the receiving unit receives the synchronization processing instruction, the receiving unit is specifically configured to:
triggering a first physical key of the intelligent video and audio equipment to generate the synchronous processing instruction;
When the receiving unit receives the synchronization processing completion instruction, the receiving unit is specifically configured to:
And triggering a second physical key of the intelligent video and audio equipment to generate the synchronous processing completion instruction.
6. A computer device, comprising:
Processor, memory, bus, input/output interface, network interface;
the processor is connected with the memory, the input/output interface and the network interface through the bus;
the memory stores a program;
The processor, when executing the program stored in the memory, realizes the synchronization processing method of the picture and the sound according to any one of claims 1 to 4.
7. A computer storage medium having stored therein instructions which, when executed on a computer, cause the computer to perform the method of synchronous processing of pictures and sounds according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211368580.0A CN115720278B (en) | 2022-11-03 | 2022-11-03 | Synchronous processing method of sound and picture and related device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211368580.0A CN115720278B (en) | 2022-11-03 | 2022-11-03 | Synchronous processing method of sound and picture and related device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115720278A CN115720278A (en) | 2023-02-28 |
| CN115720278B true CN115720278B (en) | 2024-12-31 |
Family
ID=85254667
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211368580.0A Active CN115720278B (en) | 2022-11-03 | 2022-11-03 | Synchronous processing method of sound and picture and related device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115720278B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN204305260U (en) * | 2014-12-28 | 2015-04-29 | 冠捷显示科技（厦门）有限公司 | The television set of a kind of video and wireless sound box Audio Matching |
| CN111988647A (en) * | 2020-08-27 | 2020-11-24 | 广州视源电子科技股份有限公司 | Sound and picture synchronous adjusting method, device, equipment and medium |
| CN114827679A (en) * | 2022-04-19 | 2022-07-29 | 海信视像科技股份有限公司 | Display device and sound picture synchronization method |
Family Cites Families (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4277512B2 (en) * | 2002-11-15 | 2009-06-10 | カシオ計算機株式会社 | Electronic device and program |
| CN101005589A (en) * | 2006-01-16 | 2007-07-25 | 深圳Tcl新技术有限公司 | Control method for keeping synchronization of audio-video frequencies |
| CN101674486B (en) * | 2009-09-29 | 2013-05-08 | 深圳市融创天下科技股份有限公司 | Streaming media audio and video synchronization method and system |
| WO2013082965A1 (en) * | 2011-12-05 | 2013-06-13 | 优视科技有限公司 | Streaming media data processing method and apparatus and streaming media data reproducing device |
| CN103167342B (en) * | 2013-03-29 | 2016-07-13 | 天脉聚源(北京)传媒科技有限公司 | A kind of audio-visual synchronization processing means and method |
| CN105376628B (en) * | 2014-08-27 | 2018-09-21 | 深圳Tcl新技术有限公司 | Audio and video frequency signal synchronizing method and device |
| CN106507161B (en) * | 2016-11-29 | 2019-11-15 | 腾讯科技（深圳）有限公司 | Video live broadcast method and live broadcast device |
| CN107801080A (en) * | 2017-11-10 | 2018-03-13 | 普联技术有限公司 | A kind of audio and video synchronization method, device and equipment |
| CN108449617B (en) * | 2018-02-11 | 2020-04-03 | 浙江大华技术股份有限公司 | Method and device for controlling audio and video synchronization |
| CN109167890B (en) * | 2018-08-22 | 2021-03-05 | 海信视像科技股份有限公司 | Sound and picture synchronization method and device and display equipment |
| CN111432258B (en) * | 2019-04-29 | 2022-09-30 | 杭州海康威视数字技术股份有限公司 | Audio and video signal synchronization method and device |
| CN111050207A (en) * | 2019-12-05 | 2020-04-21 | 海信电子科技（深圳）有限公司 | Television channel switching method and television |
| CN113301410A (en) * | 2021-04-30 | 2021-08-24 | 西安诺瓦星云科技股份有限公司 | Audio picture synchronization method, device and system and transmission controller |
| CN114302021B (en) * | 2021-04-30 | 2024-11-08 | 海信视像科技股份有限公司 | Display device and audio-visual synchronization method |
-
2022
- 2022-11-03 CN CN202211368580.0A patent/CN115720278B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN204305260U (en) * | 2014-12-28 | 2015-04-29 | 冠捷显示科技（厦门）有限公司 | The television set of a kind of video and wireless sound box Audio Matching |
| CN111988647A (en) * | 2020-08-27 | 2020-11-24 | 广州视源电子科技股份有限公司 | Sound and picture synchronous adjusting method, device, equipment and medium |
| CN114827679A (en) * | 2022-04-19 | 2022-07-29 | 海信视像科技股份有限公司 | Display device and sound picture synchronization method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115720278A (en) | 2023-02-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP2892239A1 (en) | Living room computer with small form-factor pluggable port | |
| EP3319320A2 (en) | Adaptive media streaming method and apparatus according to decoding performance | |
| EP3503570A1 (en) | Method of transmitting video frames from a video stream to a display and corresponding apparatus | |
| CN109167890B (en) | Sound and picture synchronization method and device and display equipment | |
| JPWO2014141425A1 (en) | Video display system, source device, sink device, and video display method | |
| US20130166769A1 (en) | Receiving device, screen frame transmission system and method | |
| CN114554277B (en) | Multimedia processing method, device, server and computer readable storage medium | |
| TWI663875B (en) | Video processing method and device thereof | |
| CN113923508A (en) | Screen projection method and device | |
| CN113691862B (en) | Video processing method, electronic device for video playing and video playing system | |
| CN115720278B (en) | Synchronous processing method of sound and picture and related device | |
| CN111741343B (en) | Video processing method and device and electronic equipment | |
| US12368917B2 (en) | Devices, systems, and methods for multi-user screen sharing | |
| US10506138B2 (en) | Driving displays at cinematic frame rates | |
| EP4202912A1 (en) | Image processing method and apparatus for virtual reality device, and virtual reality device | |
| KR102660576B1 (en) | Audio apparatus and method of controlling the same | |
| CN111416994B (en) | Method and device for synchronously presenting video stream and tracking information and electronic equipment | |
| CN116017012B (en) | Multi-screen synchronization method, device, display equipment and computer readable storage medium | |
| CN102955666B (en) | Method for automatically adjusting wireless display quality, display device and communication device thereof | |
| US20240048727A1 (en) | Method and system of low latency video coding with interactive applications | |
| CN115811628A (en) | Synchronous processing method of sound and picture and related device | |
| WO2013145699A1 (en) | Image transfer system, image transfer method, terminal apparatus, image transmission method, and non-temporary computer readable medium storing image transmission program | |
| CN119729126A (en) | Method and device for solving tearing of cloud game scene picture based on self-adaptive synchronization technology | |
| CN117939213A (en) | Display device, multi-window display method, and storage medium | |
| CN117193689A (en) | Multi-screen display method, system and related equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |