CN113082710B - Audio playing method and device, electronic equipment and storage medium - Google Patents
Audio playing method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN113082710B CN113082710B CN202110441066.4A CN202110441066A CN113082710B CN 113082710 B CN113082710 B CN 113082710B CN 202110441066 A CN202110441066 A CN 202110441066A CN 113082710 B CN113082710 B CN 113082710B
- Authority
- CN
- China
- Prior art keywords
- audio
- frame
- frames
- audio frames
- sound card
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/50—Controlling the output signals based on the game progress
- A63F13/54—Controlling the output signals based on the game progress involving acoustic signals, e.g. for simulating revolutions per minute [RPM] dependent engine sounds in a driving game or reverberation against a virtual wall
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/16—Sound input; Sound output
- G06F3/165—Management of the audio stream, e.g. setting of volume, audio stream path
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/16—Sound input; Sound output
- G06F3/167—Audio in a user interface, e.g. using voice commands for navigating, audio feedback
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- General Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Telephonic Communication Services (AREA)
Abstract
The invention discloses an audio playing method, an audio playing device, electronic equipment and a storage medium, wherein the method comprises the following steps: when starting audio processing, determining the first audio frame number of the sound card cache and the second audio frame number of the audio source equipment sent to the audio software cache; calculating the total audio frame number by using the first audio frame number and the second audio frame number, and carrying out frame loss operation on the audio frames in the audio software cache when the total audio frame number is greater than a first threshold value; when the frame loss operation is completed and the number of the first audio frames is smaller than a second threshold value, transmitting the audio frames in the audio software buffer to a sound card buffer; after the sending operation is finished, ending the audio processing, and playing by utilizing the audio frames in the sound card buffer; the method can effectively control the total audio quantity, alleviate audio playing delay, ensure that the sound card caches enough audio frames for playing, and effectively improve the playing experience of the cloud game audio.
Description
Technical Field
The present invention relates to the field of audio processing, and in particular, to an audio playing method, an audio playing device, an electronic device, and a storage medium.
Background
With the continuous development of streaming technology and cloud computer technology, the cloud game can run the game on the remote terminal, and the game operation is performed on the local terminal through the streaming technology, so that the effect of playing the high-configuration game on the local terminal with lower configuration is achieved. In the related art, audio data of a cloud game needs to be generated by a remote terminal and sent to audio software of a local terminal, and is played by a sound card of the local terminal, however, the local terminal is affected by network congestion, and the local terminal cannot receive audio data packets or receives excessive audio data packets, so that the sound card has no audio data to be played, or the sound card needs to process a large number of audio data packets, which causes playing delay and affects user experience of the cloud game.
Disclosure of Invention
The invention aims to provide an audio playing method, an audio playing device, electronic equipment and a storage medium, which can effectively control the total audio quantity, relieve audio playing delay, ensure that a sound card caches enough audio frames for playing, and finally effectively promote the playing experience of cloud game audio.
In order to solve the above technical problems, the present invention provides an audio playing method, including:
When starting audio processing, determining the first audio frame number of the sound card cache and the second audio frame number of the audio source equipment sent to the audio software cache;
Calculating the total audio frame number by using the first audio frame number and the second audio frame number, and performing frame loss operation on the audio frames in the audio software cache when the total audio frame number is greater than a first threshold;
When the frame dropping operation is completed and the number of the first audio frames is smaller than a second threshold value, a sending operation of sending the audio frames in the audio software buffer memory to the sound card buffer memory is performed;
And after the sending operation is finished, ending the audio processing, and playing by utilizing the audio frames in the sound card cache.
Optionally, the performing a frame dropping operation on the audio frames in the audio software buffer includes:
According to the receiving sequence of the audio frames in the audio software buffer, the audio frames received at the latest in the audio software buffer are subjected to frame loss, and the second audio frame number and the total audio frame number are updated;
judging whether the second audio frame number reaches an audio frame retention value or not;
if yes, judging that the frame loss operation is completed;
if not, continuing to execute the step of frame loss of the audio frames received at the latest in the audio software buffer according to the receiving sequence of the audio frames in the audio software buffer until the total number of the audio frames is less than or equal to the first threshold value, and judging that the frame loss operation is completed.
Optionally, the sending operation of sending the audio frames in the audio software buffer to the sound card buffer includes:
according to the receiving sequence, the earliest received audio frame in the audio software buffer is sent, and the second audio frame number and the audio frame sending number are updated;
judging whether the second audio frame number reaches the audio frame retention value;
if yes, judging that the sending operation is completed;
if not, continuing to execute the step of transmitting the earliest received audio frame in the audio software buffer according to the receiving sequence until the transmission number of the audio frames is equal to a third threshold value, and judging that the transmission operation is completed.
Optionally, the ending the audio processing includes:
Recording the end time and setting the first audio frame number in the sound card buffer as the historical audio frame number;
correspondingly, when the audio processing is started, determining the first audio frame number cached by the sound card includes:
determining a starting time, and calculating the playing quantity of the audio frames from the ending time to the starting time by utilizing pre-stored audio frame format data;
And calculating the first audio frame number by using the historical audio frame number and the audio frame playing number.
Optionally, before starting the audio processing, further comprising:
And when the connection is established with the audio source equipment, acquiring the audio frame format data from the audio source equipment.
The invention also provides an audio playing device, which comprises:
The number determining module is used for determining the number of first audio frames buffered by the sound card and the number of second audio frames transmitted to the audio software buffer by the audio source equipment when the audio processing is started;
The frame loss module is used for calculating the total audio frame number by utilizing the first audio frame number and the second audio frame number, and carrying out frame loss operation on the audio frames in the audio software cache when the total audio frame number is greater than a first threshold value;
The sending module is used for sending the audio frames in the audio software buffer to the sound card buffer when the frame dropping operation is completed and the number of the first audio frames is smaller than a second threshold value;
and the playing module is used for ending the audio processing after the sending operation is completed and playing by utilizing the audio frames in the sound card cache.
Optionally, the frame loss module includes:
The frame loss sub-module is used for carrying out frame loss on the latest received audio frames in the audio software buffer according to the receiving sequence of the audio frames in the audio software buffer, and updating the second audio frame number and the total audio frame number;
the first judging module is used for judging whether the second audio frame number reaches an audio frame retention value or not;
the first processing module is used for judging that the frame loss operation is completed if yes;
and the second processing module is used for continuously executing the step of frame loss of the audio frames received at the latest in the audio software buffer according to the receiving sequence of the audio frames in the audio software buffer if not, and judging that the frame loss operation is completed when the total number of the audio frames is smaller than or equal to the first threshold value.
Optionally, the sending module includes:
the sending sub-module is used for sending the earliest received audio frames in the audio software buffer according to the receiving sequence, and updating the second audio frame number and the audio frame sending number;
a second judging sub-module, configured to judge whether the second number of audio frames reaches the audio frame reserved value;
the third processing module is used for judging that the sending operation is finished if yes;
And the fourth processing module is used for continuously executing the step of transmitting the earliest received audio frames in the audio software buffer according to the receiving sequence if not, and judging that the transmission operation is completed when the transmission number of the audio frames is equal to a third threshold value.
The present invention also provides an electronic device including:
a memory for storing a computer program;
And a processor for implementing the audio playing method as described above when executing the computer program.
The present invention also provides a storage medium having stored therein computer executable instructions which, when loaded and executed by a processor, implement an audio playing method as described above.
The invention provides an audio playing method, which comprises the following steps: when starting audio processing, determining the first audio frame number of the sound card cache and the second audio frame number of the audio source equipment sent to the audio software cache; calculating the total audio frame number by using the first audio frame number and the second audio frame number, and performing frame loss operation on the audio frames in the audio software cache when the total audio frame number is greater than a first threshold; when the frame dropping operation is completed and the number of the first audio frames is smaller than a second threshold value, a sending operation of sending the audio frames in the audio software buffer memory to the sound card buffer memory is performed; and after the sending operation is finished, ending the audio processing, and playing by utilizing the audio frames in the sound card cache.
Therefore, the method firstly determines the total audio number in the sound card cache and the audio software cache, and when the total audio number is larger than the first threshold value, carries out frame loss operation on the audio frames in the audio cache, and can effectively control the total audio number in the sound card cache and the audio software cache, thereby ensuring that the sound card can efficiently process the playing of the audio frames and avoiding the interference of the received excessive audio data packets on the audio playing; meanwhile, after the frame loss operation is completed, the method also judges the number of the first audio frames in the sound card cache, and when the number of the first audio frames is smaller than a second threshold value, the audio frames in the audio software cache are sent to the sound card cache, so that the sound card cache can be effectively ensured to always store enough audio frames, finally, the delay influence of network blocking on audio playing can be effectively relieved, the influence of no audio data in the sound card cache on audio playing can be effectively relieved, and the game audio playing experience can be effectively improved. The invention also provides an audio playing device, electronic equipment and a storage medium, which have the beneficial effects.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only embodiments of the present invention, and that other drawings can be obtained according to the provided drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of an audio playing method according to an embodiment of the present invention;
fig. 2 is a schematic diagram of corresponding time of audio processing according to an embodiment of the present invention;
fig. 3 is a block diagram of an audio playing device according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
In the related art, audio data of a cloud game needs to be generated by a remote terminal and sent to audio software of a local terminal, and is played by a sound card of the local terminal, however, the local terminal is affected by network congestion, and the local terminal cannot receive audio data packets or receives excessive audio data packets, so that the sound card has no audio data to be played, or the sound card needs to process a large number of audio data packets, which causes playing delay and affects user experience of the cloud game. In view of this, the present invention provides an audio playing method, which can effectively control the total audio number, alleviate audio playing delay, ensure that a sound card caches enough audio frames for playing, and finally effectively promote the playing experience of cloud game audio. Referring to fig. 1, fig. 1 is a flowchart of an audio playing method according to an embodiment of the present invention, where the method may include:
s101, when audio processing is started, determining the first audio frame number of the sound card buffer memory and the second audio frame number of the audio source equipment transmitted to the audio software buffer memory.
It should be noted that, the embodiment of the present invention is not limited to a specific manner of determining the number of the first audio frames in the sound card buffer or the number of the second audio frames in the audio software buffer, for example, the sound card buffer and the audio frames in the audio software buffer may be counted to obtain the corresponding number of the audio frames, or an independent counter may be maintained, and when the number of the audio frames changes (for example, frame loss, sending, playing, etc.), the counter may be updated in real time. Considering that the first way of determining the number of audio frames requires statistics each time audio processing is performed, the efficiency of audio statistics will be reduced, and the way of maintaining the counter will enable an efficient determination of the number of audio frames, the number of audio frames may be determined by maintaining the counter in embodiments of the present invention. Further, considering that the audio software buffer is used for storing the audio frame data transmitted by the audio source device, when the audio frame data is transmitted, the stable transmission of the audio frame data cannot be ensured, that is, the transmission rate of the audio frame data cannot be determined due to the influence of network factors. In this case, the determination of the number of audio frames in the audio software buffer by maintaining the counter is not accurate, so that the second number of audio frames may be determined for the audio software buffer by sending a query to the audio software buffer or by counting directly in the audio software buffer. For the sound card buffer, the sending action of sending the audio frames in the audio software buffer to the sound card buffer is a controllable action in the embodiment of the invention, namely, the number of the sent audio frames can be specifically controlled, so that the number of the first audio frames in the sound card buffer can be rapidly and accurately determined by using a mode of maintaining a counter, and further, for the sound card buffer, the number of the first audio frames can be determined by using a mode of maintaining the counter.
Further, it should be noted that, at the time of starting the audio processing, the audio frames in the audio software buffer are already audio frames for which the decoding operation is completed. The embodiment of the present invention is not limited to a specific decoding method, and specific reference may be made to the related art of decoding. The embodiment of the invention also does not limit the specific mode and receiving mode of the audio source equipment for transmitting the audio frame data, and can refer to related technologies such as cloud computers, audio streaming and the like.
S102, calculating the total audio frame number by using the first audio frame number and the second audio frame number, and carrying out frame loss operation on the audio frames in the audio software buffer when the total audio frame number is greater than a first threshold value.
In the embodiment of the invention, in order to ensure that the sound card can efficiently process the playing of the audio frames, the total audio frame number is obtained by summing the first audio frame number and the second audio frame number, and when the audio frame number is greater than the first threshold value, the frame losing operation is carried out on the audio frames in the audio software cache, so that the total number of the audio frames in the sound card cache and the audio software cache can be effectively controlled, further, the situation that excessive audio frames which can cause delay influence on the playing are not generated in the two caches can be ensured, and the delay influence on the playing caused by the fact that the audio software receives excessive audio frame data due to network blocking can be effectively relieved. It should be noted that, the embodiment of the present invention is not limited to the specific value of the first threshold, and may be set according to the actual application requirement.
Furthermore, the embodiment of the invention is not limited to specific frame dropping operation, for example, the number of dropped frames can be determined first, and the audio frames with the number of dropped frames can be directly dropped, or of course, single-frame audio frames can be circularly dropped, and the number of second audio frames and the number of total audio frames are continuously updated in each cycle until the number of total audio frames is less than or equal to the first threshold value, and the frame dropping operation is judged to be completed. In order to more accurately perform the frame dropping operation, in the embodiment of the present invention, the frame dropping operation may be performed by circularly dropping a single-frame audio frame. It will be appreciated that the audio frames received the latest in the audio software buffer may be dropped according to the order in which the audio frames were received. Further, considering that the audio frames of the single-frame audio frame to be discarded are from the audio software buffer, and the exit condition is that the total number of audio frames is less than or equal to the first threshold, however, there are cases that the audio frames in the audio software buffer are less and the audio frames in the sound card buffer are more, that is, even if all the audio frames in the audio software buffer are discarded, the total number of audio frames cannot be determined to be less than or equal to the first threshold, at this time, in each cycle, the second number of audio frames in the audio software buffer can be determined, and when the second number of audio frames is 0, the frame discarding operation is determined to be completed. Of course, in order to ensure that the audio can be spliced effectively, an audio frame retention value may also be set to ensure that at least the number of audio frames with the audio frame retention value can be retained in the audio software buffer, in other words, in each cycle, the number of second audio frames in the audio software buffer may be determined, and when the number of second audio frames reaches the audio retention value, the frame dropping operation is determined to be completed. It should be noted that, the embodiment of the present invention is not limited to a specific audio reservation value, and may be set according to practical application requirements, in one case, the audio reservation value may be set to 1, that is, at least one audio frame is stored in the audio software buffer.
In one possible scenario, performing a frame dropping operation on an audio frame in an audio software buffer includes:
Step 11: according to the receiving sequence of the audio frames in the audio software buffer, the audio frames received at the latest in the audio software buffer are subjected to frame loss, and the second audio frame number and the total audio frame number are updated;
step 12: judging whether the second audio frame number reaches an audio frame retention value or not; if yes, go to step 13; if not, go to step 14;
step 13: judging that the frame loss operation is completed;
Step 14: and continuing to execute the step of frame loss of the latest received audio frames in the audio software buffer according to the receiving sequence of the audio frames in the audio software buffer until the total number of the audio frames is less than or equal to a first threshold value, and judging that the frame loss operation is completed.
It can be understood that after the audio frames received the latest in the audio software buffer are lost, the number of second audio frames and the total number of audio frames can be reduced by one to complete the number update.
And S103, when the frame loss operation is completed and the number of the first audio frames is smaller than a second threshold value, transmitting the audio frames in the audio software buffer to the sound card buffer.
In the embodiment of the invention, in order to ensure that a sufficient number of audio frames available for playing exist in the sound card cache, after the frame loss operation is completed, the number of the first audio frames is also determined, and when the number of the first audio frames is smaller than a second threshold value representing the playing minimum limit, the audio frames in the audio software cache are sent to the sound card cache. It should be noted that, the embodiment of the present invention is not limited to a specific value of the second threshold, and the user may set the value according to the actual application requirement.
Further, the embodiment of the present invention is not limited to a specific transmission operation, for example, the transmission number may be determined first, and the audio frames of the transmission number may be directly transmitted, or a single-frame audio frame may be cyclically transmitted. In order to perform the transmission operation more accurately, in the embodiment of the present invention, the transmission operation may be performed by cyclically transmitting a single-frame audio frame. It will be appreciated that the earliest received audio frame in the audio software buffer may be transmitted in accordance with the order in which the audio frames were received. Furthermore, the embodiment of the present invention does not limit the exit condition for exiting the single-frame audio frame transmission cycle, for example, the number of second audio frames may be updated in real time, and the cycle may be exited when the number of second audio frames reaches the audio reservation value or is a space; of course, it is also possible to count the number of audio frame transmissions and exit the loop when the number of audio frame transmissions reaches a third threshold value representing an upper audio play limit. In order to ensure that the amount of audio frames in the sound card buffer is appropriate while ensuring that the necessary audio frames are retained in the audio software buffer, in an embodiment of the present invention, the exit condition for exiting the single-frame audio frame transmission loop may be that the second number of audio frames reaches the audio retention value or that the number of audio frames transmitted reaches a third threshold value representing an upper audio playback limit. It should be noted that, the embodiment of the present invention is not limited to the specific value of the third threshold, and may be set according to the actual application requirement. Further, it will be appreciated that the earliest received audio frame in the audio software buffer may be transmitted in accordance with the order in which the audio frames were received.
In one possible case, the sending operation of sending the audio frames in the audio software buffer to the sound card buffer is performed, including:
step 21: according to the receiving sequence, the earliest received audio frame in the audio software buffer is sent, and the second audio frame number and the audio frame sending number are updated;
Step 22: judging whether the second audio frame number reaches an audio frame retention value or not; if yes, go to step 23; if not, go to step 24;
Step 23: then it is determined that the transmission operation is completed;
Step 24: and continuing to execute the step of transmitting the earliest received audio frames in the audio software buffer according to the receiving sequence until the transmission number of the audio frames is equal to a third threshold value, and judging that the transmission operation is completed.
And S104, after the sending operation is finished, ending the audio processing, and playing by utilizing the audio frames in the sound card cache.
It should be noted that there is a time interval between each end of the audio processing and the start of the next audio processing, and in this time interval the sound card plays with the audio frames in the sound card buffer, i.e. the number of first audio frames in the sound card buffer is reduced. When the number of first audio frames is determined by maintaining a counter, the number of audio plays between the time intervals needs to be calculated in order to ensure that the number of first audio frames is accurate.
In one possible case, ending the audio processing includes:
step 31: recording the end time and setting the first audio frame number in the sound card buffer as the historical audio frame number.
Accordingly, when starting audio processing, determining the number of first audio frames buffered by the sound card includes:
Step 41: determining a starting time, and calculating the playing quantity of the audio frames from the ending time to the starting time by utilizing pre-stored audio frame format data;
Step 42: and calculating the first audio frame number by using the historical audio frame number and the audio frame playing number.
It should be noted that, the embodiment of the present invention is not limited to specific audio frame format data, as long as the audio frame format data can calculate the number of audio frames played in a unit time, and further can determine the number of audio frames played at the beginning of the ending time value. In one possible scenario, the audio frame format data may include audio channel number, audio bit depth, audio sampling rate calculation, and the number of audio frames played per unit time l=audio channel number, audio bit depth, audio sampling rate per unit time.
Furthermore, it should be noted that the embodiment of the present invention is not limited to the setting manner of the audio frame format data, for example, the audio frame format data may be manually set and stored locally, or may be obtained from an audio source device. The audio frame format data may be set by way of acquisition to the audio source device, considering that different audio source devices may carry different audio frame format data.
In one possible case, before starting the audio processing, further comprising:
step 51: when a connection is established with an audio source device, audio frame format data is acquired from the audio source device.
Based on the above embodiment, the method first determines the total audio number in the sound card cache and the audio software cache, and when the total audio number is greater than the first threshold, performs a frame dropping operation on the audio frames in the audio cache, so as to effectively control the total audio number in the sound card cache and the audio software cache, further ensure that the sound card can efficiently process the playing of the audio frames, and avoid the interference of receiving excessive audio data packets on the playing of the audio; meanwhile, after the frame loss operation is completed, the method also judges the number of the first audio frames in the sound card cache, and when the number of the first audio frames is smaller than a second threshold value, the audio frames in the audio software cache are sent to the sound card cache, so that the sound card cache can be effectively ensured to always store enough audio frames, finally, the delay influence of network blocking on audio playing can be effectively relieved, the influence of no audio data in the sound card cache on audio playing can be effectively relieved, and the game audio playing experience can be effectively improved.
The above-described audio playback method is specifically described below based on a specific time axis diagram. Referring to fig. 2, fig. 2 is a schematic diagram of corresponding time of audio processing according to an embodiment of the invention. For the ith audio processing, the specific flow is as follows:
1. in the ith processing cycle, the time when the host starts the frame dropping operation is recorded as The time at which the transmission operation is started is denoted as/>The time at which playback starts is denoted/>Time of day. Time of starting playing in the (i-1) -th processing cycle/>Time/>, by the beginning of frame loss in the ith processing cycleThe audio play amount between is/>And L represents the audio play amount in unit time, and is obtained from the host end (i.e. the audio source device) of the cloud computer system after connection is established with the host end, wherein l=audio channel number, audio bit depth, audio sampling rate and unit time.
2. The frame loss threshold of the audio buffer is set to be T1 (first threshold), the lower threshold of the audio playing is set to be T2 (second threshold), and the higher threshold of the audio playing is set to be T3 (third threshold).
3. The audio software cache in the system is set as cache A, and the sound card cache is set as cache B. Data in cache a is denoted DA and data in cache B is denoted DB. In the ith processing cycle, the data amount in the buffer B at the beginning of frame loss is recorded asCalculated as/>In the ith processing cycle, the data amount in the buffer A at the beginning of frame loss is recorded asIf/>And then the data in the buffer A is subjected to frame loss, and a frame loss cycle is started. In the frame loss cycle, after each frame loss, the data volume in the buffer A is updated to/>And judgeIf so, continuing to lose the frame. In the frame loss process, at least one frame is reserved in the buffer A.
4. Obtaining the frame number in the buffer A after the frame loss is completed, and recording asIf/>The data in the audio buffer is not fetched into the sound card.
5. If it isThe data in cache a is fetched into cache B. In the frame taking cycle, after each frame taking, the update in the buffer A is/>The frame number calculation method is/>The number of frames of the data frame is at most T3 frames, and at least one frame is reserved in the buffer A.
6. And after the ith processing cycle is finished, playing according to the received audio data in the sound card.
The following describes an audio playing device, an electronic device, and a storage medium according to embodiments of the present invention, where the audio playing device, the electronic device, and the storage medium described below may be referred to correspondingly with the audio playing method described above.
Referring to fig. 3, fig. 3 is a block diagram of an audio playing device according to an embodiment of the present invention, where the device includes:
A number determining module 301, configured to determine, when audio processing is started, a first number of audio frames buffered by the sound card and a second number of audio frames transmitted to the audio software buffer by the audio source device;
The frame loss module 302 is configured to calculate a total number of audio frames by using the first number of audio frames and the second number of audio frames, and perform a frame loss operation on the audio frames in the audio software buffer when the total number of audio frames is greater than a first threshold;
a sending module 303, configured to perform a sending operation of sending the audio frames in the audio software buffer to the sound card buffer when the frame dropping operation is completed and the number of the first audio frames is less than the second threshold;
and the playing module 304 is configured to end the audio processing after the sending operation is completed, and play the audio frame in the sound card buffer.
Optionally, the frame dropping module 302 may include:
The frame loss submodule is used for carrying out frame loss on the audio frames received at the latest in the audio software buffer according to the receiving sequence of the audio frames in the audio software buffer, and updating the second audio frame number and the total audio frame number;
the first judging module is used for judging whether the number of the second audio frames reaches an audio frame reserved value or not;
The first processing module is used for judging that the frame loss operation is completed if yes;
And the second processing module is used for continuously executing the step of frame loss of the audio frames received at the latest in the audio software buffer according to the receiving sequence of the audio frames in the audio software buffer if not, and judging that the frame loss operation is completed when the total number of the audio frames is smaller than or equal to a first threshold value.
Optionally, the sending module 303 may include:
The sending sub-module is used for sending the earliest received audio frames in the audio software buffer according to the receiving sequence, and updating the second audio frame number and the audio frame sending number;
the second judging sub-module is used for judging whether the second audio frame number reaches an audio frame retention value or not;
The third processing module is used for judging that the sending operation is finished if yes;
and the fourth processing module is used for continuously executing the step of transmitting the earliest received audio frames in the audio software buffer according to the receiving sequence if not, and judging that the transmission operation is completed until the transmission number of the audio frames is equal to a third threshold value.
Optionally, the playing module 304 may include:
The end time setting sub-module is used for recording the end time and setting the number of the first audio frames in the sound card buffer as the number of the historical audio frames;
Accordingly, the number determining module 301 may include:
A start time setting sub-module, configured to determine a start time, and calculate a number of audio frames played from an end time to the start time by using pre-stored audio frame format data;
and the number determination submodule is used for calculating the number of the first audio frames by utilizing the number of the historical audio frames and the playing number of the audio frames.
Optionally, the apparatus may further include:
And the acquisition module is used for acquiring the audio frame format data from the audio source equipment when the connection is established with the audio source equipment.
The embodiment of the invention also provides electronic equipment, which comprises:
a memory for storing a computer program;
and a processor for implementing the steps of the audio playing method as described above when executing the computer program.
Since the embodiment of the electronic device portion corresponds to the embodiment of the audio playing method portion, the embodiment of the electronic device portion is referred to the description of the embodiment of the audio playing method portion, and is not repeated herein.
The embodiment of the invention also provides a storage medium, and a computer program is stored on the storage medium, and when the computer program is executed by a processor, the steps of the audio playing method in any embodiment are realized.
Since the embodiments of the storage medium portion and the embodiments of the audio playing method portion correspond to each other, the embodiments of the storage medium portion are referred to for description of the embodiments of the audio playing method portion, and are not repeated herein.
In the description, each embodiment is described in a progressive manner, and each embodiment is mainly described by the differences from other embodiments, so that the same similar parts among the embodiments are mutually referred. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative elements and steps are described above generally in terms of functionality in order to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. The software modules may be disposed in Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
The audio playing method, the audio playing device, the electronic equipment and the storage medium provided by the invention are described in detail. The principles and embodiments of the present invention have been described herein with reference to specific examples, the description of which is intended only to facilitate an understanding of the method of the present invention and its core ideas. It should be noted that it will be apparent to those skilled in the art that various modifications and adaptations of the invention can be made without departing from the principles of the invention and these modifications and adaptations are intended to be within the scope of the invention as defined in the following claims.
Claims (6)
1. An audio playing method, comprising:
When starting audio processing, determining the first audio frame number of the sound card cache and the second audio frame number of the audio source equipment sent to the audio software cache;
Calculating the total audio frame number by using the first audio frame number and the second audio frame number, and performing frame loss operation on the audio frames in the audio software cache when the total audio frame number is greater than a first threshold;
When the frame dropping operation is completed and the number of the first audio frames is smaller than a second threshold value, a sending operation of sending the audio frames in the audio software buffer memory to the sound card buffer memory is performed, and when the number of the audio frames sent is equal to a third threshold value, the sending operation is judged to be completed; the number of the audio frames is the number of the audio frames sent from the audio software cache to the sound card cache;
after the sending operation is completed, ending the audio processing, and playing by utilizing the audio frames in the sound card cache;
the frame dropping operation for the audio frames in the audio software cache comprises the following steps:
According to the receiving sequence of the audio frames in the audio software buffer, the audio frames received at the latest in the audio software buffer are subjected to frame loss, and the second audio frame number and the total audio frame number are updated;
judging whether the second audio frame number reaches an audio frame retention value or not;
if yes, judging that the frame loss operation is completed;
if not, continuing to execute the step of frame loss of the audio frames received at the latest in the audio software buffer according to the receiving sequence of the audio frames in the audio software buffer until the total number of the audio frames is less than or equal to the first threshold value, and judging that the frame loss operation is completed;
The sending operation of sending the audio frames in the audio software buffer to the sound card buffer includes:
according to the receiving sequence, the earliest received audio frame in the audio software buffer is sent, and the second audio frame number and the audio frame sending number are updated;
judging whether the second audio frame number reaches the audio frame retention value;
if yes, judging that the sending operation is completed;
if not, continuing to execute the step of transmitting the earliest received audio frame in the audio software buffer according to the receiving sequence until the transmission number of the audio frames is equal to a third threshold value, and judging that the transmission operation is completed.
2. The audio playing method according to claim 1, wherein the ending the audio processing includes:
Recording the end time and setting the first audio frame number in the sound card buffer as the historical audio frame number;
correspondingly, when the audio processing is started, determining the first audio frame number cached by the sound card includes:
determining a starting time, and calculating the playing quantity of the audio frames from the ending time to the starting time by utilizing pre-stored audio frame format data;
And calculating the first audio frame number by using the historical audio frame number and the audio frame playing number.
3. The audio playing method according to claim 2, further comprising, before starting the audio processing:
And when the connection is established with the audio source equipment, acquiring the audio frame format data from the audio source equipment.
4. An audio playback apparatus, comprising:
The number determining module is used for determining the number of first audio frames buffered by the sound card and the number of second audio frames transmitted to the audio software buffer by the audio source equipment when the audio processing is started;
The frame loss module is used for calculating the total audio frame number by utilizing the first audio frame number and the second audio frame number, and carrying out frame loss operation on the audio frames in the audio software cache when the total audio frame number is greater than a first threshold value;
the sending module is used for sending the audio frames in the audio software buffer to the sound card buffer when the frame dropping operation is completed and the number of the first audio frames is smaller than a second threshold value, and judging that the sending operation is completed when the number of the audio frames is equal to a third threshold value; the number of the audio frames is the number of the audio frames sent from the audio software cache to the sound card cache;
The playing module is used for ending the audio processing after the sending operation is completed and playing by utilizing the audio frames in the sound card cache;
the frame loss module comprises:
The frame loss sub-module is used for carrying out frame loss on the latest received audio frames in the audio software buffer according to the receiving sequence of the audio frames in the audio software buffer, and updating the second audio frame number and the total audio frame number;
the first judging module is used for judging whether the second audio frame number reaches an audio frame retention value or not;
the first processing module is used for judging that the frame loss operation is completed if yes;
the second processing module is used for continuously executing the step of frame loss of the audio frames received at the latest in the audio software buffer according to the receiving sequence of the audio frames in the audio software buffer if not, and judging that the frame loss operation is completed when the total number of the audio frames is smaller than or equal to the first threshold value;
The sending module comprises:
the sending sub-module is used for sending the earliest received audio frames in the audio software buffer according to the receiving sequence, and updating the second audio frame number and the audio frame sending number;
a second judging sub-module, configured to judge whether the second number of audio frames reaches the audio frame reserved value;
the third processing module is used for judging that the sending operation is finished if yes;
And the fourth processing module is used for continuously executing the step of transmitting the earliest received audio frames in the audio software buffer according to the receiving sequence if not, and judging that the transmission operation is completed when the transmission number of the audio frames is equal to a third threshold value.
5. An electronic device, comprising:
a memory for storing a computer program;
A processor for implementing the audio playing method according to any one of claims 1 to 3 when executing the computer program.
6. A storage medium having stored therein computer executable instructions which, when loaded and executed by a processor, implement the audio playing method of any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110441066.4A CN113082710B (en) | 2021-04-23 | 2021-04-23 | Audio playing method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110441066.4A CN113082710B (en) | 2021-04-23 | 2021-04-23 | Audio playing method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113082710A CN113082710A (en) | 2021-07-09 |
| CN113082710B true CN113082710B (en) | 2024-05-28 |
Family
ID=76679699
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110441066.4A Active CN113082710B (en) | 2021-04-23 | 2021-04-23 | Audio playing method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113082710B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114464211A (en) * | 2022-02-16 | 2022-05-10 | 深圳市瑞云科技有限公司 | Method for testing audio quality of cloud game |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102362269A (en) * | 2008-12-05 | 2012-02-22 | 社会传播公司 | real-time kernel |
| CN104113777A (en) * | 2014-08-01 | 2014-10-22 | 广州金山网络科技有限公司 | Audio stream decoding method and device |
| CN106775551A (en) * | 2016-10-31 | 2017-05-31 | 乐视控股（北京）有限公司 | Audio frequency playing method and system |
| CN107393556A (en) * | 2017-07-17 | 2017-11-24 | 京东方科技集团股份有限公司 | A kind of method and device for realizing audio frequency process |
| CN110007892A (en) * | 2014-04-11 | 2019-07-12 | 微软技术许可有限责任公司 | Audio Signal Processing |
| CN110248204A (en) * | 2019-07-16 | 2019-09-17 | 广州虎牙科技有限公司 | A kind of processing method, device, equipment and the storage medium of live streaming caching |
| CN112206520A (en) * | 2020-10-21 | 2021-01-12 | 深圳市欢太科技有限公司 | Real-time audio collection method, system, server, client and storage medium |
-
2021
- 2021-04-23 CN CN202110441066.4A patent/CN113082710B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102362269A (en) * | 2008-12-05 | 2012-02-22 | 社会传播公司 | real-time kernel |
| CN110007892A (en) * | 2014-04-11 | 2019-07-12 | 微软技术许可有限责任公司 | Audio Signal Processing |
| CN104113777A (en) * | 2014-08-01 | 2014-10-22 | 广州金山网络科技有限公司 | Audio stream decoding method and device |
| WO2016015670A1 (en) * | 2014-08-01 | 2016-02-04 | 广州金山网络科技有限公司 | Audio stream decoding method and device |
| CN106775551A (en) * | 2016-10-31 | 2017-05-31 | 乐视控股（北京）有限公司 | Audio frequency playing method and system |
| CN107393556A (en) * | 2017-07-17 | 2017-11-24 | 京东方科技集团股份有限公司 | A kind of method and device for realizing audio frequency process |
| CN110248204A (en) * | 2019-07-16 | 2019-09-17 | 广州虎牙科技有限公司 | A kind of processing method, device, equipment and the storage medium of live streaming caching |
| CN112206520A (en) * | 2020-10-21 | 2021-01-12 | 深圳市欢太科技有限公司 | Real-time audio collection method, system, server, client and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113082710A (en) | 2021-07-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110198495B (en) | Method, device, equipment and storage medium for downloading and playing video | |
| US9167007B2 (en) | Stream complexity mapping | |
| CN107371061A (en) | A kind of video stream playing method, device and equipment | |
| CN111225209A (en) | Video data plug flow method, device, terminal and storage medium | |
| US20150350369A1 (en) | Method For Reducing Pre-Fetching Of Multimedia Streaming Data With Minimal Impact On Playback User Experience | |
| CN113082710B (en) | Audio playing method and device, electronic equipment and storage medium | |
| CN114040257B (en) | Self-adaptive video stream transmission playing method, device, equipment and storage medium | |
| CN114979799A (en) | Panoramic video processing method, device, equipment and storage medium | |
| CN107333163A (en) | A kind of method for processing video frequency and device, a kind of terminal and storage medium | |
| CN113364767B (en) | Streaming media data display method and device, electronic equipment and storage medium | |
| CN111552454B (en) | Audio playing method, device, terminal and storage medium | |
| EP3247124B1 (en) | Client, streaming media data receiving method and streaming media data transmission system | |
| CN106303754A (en) | A kind of audio data play method and device | |
| CN118573961A (en) | Multimedia data playing method, device, equipment, readable storage medium and product | |
| CN114885209A (en) | Live data processing method, computing device and readable storage medium | |
| US8976758B2 (en) | Wireless communication system and method for adjusting used depth of buffer unit in wireless communication system | |
| CN116017046B (en) | Video processing method, device, equipment and storage medium | |
| CN113409801B (en) | Noise processing method, system, medium and device for real-time audio stream playing | |
| CN117395463A (en) | Video playing method and device | |
| US9319458B2 (en) | Site-based server selection | |
| CN112732105B (en) | Mouse jitter control method and device | |
| CN107087210A (en) | The method and terminal of video broadcasting condition are judged based on cache-time | |
| CN110933516A (en) | Multimedia live broadcast method, device and equipment | |
| CN115589506A (en) | MP4 format audio/video file generation method, system and computer storage medium | |
| CN106775551A (en) | Audio frequency playing method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |