Frame buffer merging method based on dirty rectangular region calculation
Technical Field
The invention belongs to the technical field of image processing, and particularly relates to a frame buffer merging method based on dirty rectangular region calculation.
Background
With the increasing requirements of the embedded field on graphic images, richer graphic controls are adopted to draw a software application interface, and higher requirements are placed on the frame rate of graphic display and the display quality of images. In the existing graphical user interface development framework, a dirty rectangle mechanism is generally used when the image data is interacted with the frame buffer device, so that the resource consumption is reduced by reducing the updating amount of the data. However, the dirty rectangle mechanism cannot solve the problem of screen tearing, and in a scene with frequently updated images, we can easily see abnormal phenomena such as dislocation, discontinuity and the like of images displayed on the screen. Therefore, we introduce a vertical synchronization mechanism in the gui, limit the frame buffer read by the current hardware from being overwritten by the gui at the same time, and provide the gui with other operable frame buffers through multiple buffers to reduce display delay.
However, no bond is formed between the existing dirty rectangle technique and the vertical synchronization technique, and a typical graphical user interface only updates data onto a fixed frame buffer when applying the dirty rectangle, without addressing the image merging problem between frame buffers.
In addition, technical terms commonly used in the art include:
Dirty rectangle, an important concept of graphics performance optimization, refers to the local area that changes and needs to be redrawn when the picture image is updated. The data volume of image update can be reduced by using the dirty rectangle, the resource consumption is reduced, and the display performance is enhanced.
Frame buffering device-a frame buffering device is an abstraction of graphics hardware, representing the frame buffer area of some video hardware and allowing software applications to access the graphics hardware through well-defined interfaces.
And (3) vertical synchronization, namely, when the screen of the display device reads the frame buffer row by row, the frame buffer currently used by hardware is forbidden to be updated again by the image, and the image is not allowed to overwrite the current frame buffer until the current frame buffer is not read any more after the screen is displayed, so that the problem of tearing the screen is avoided, but display delay is caused.
When the display device screen reads the images in the frame buffer line by line for display, the program is drawing the picture and overwrites the read frame buffer, so that abnormal phenomena such as dislocation and discontinuity of the images displayed on the screen occur.
Multiple buffering when the display device enables the vertical synchronization mechanism, the number of frame buffers can be increased to provide more buffering for image updates in order to reduce display delay. Thus, when one frame buffer is in a disabled state, the image can be overlaid on other frame buffers, and when the screen reading of the display device completes one frame buffer, the other frame buffer which is updated can be switched in time for reading.
Disclosure of Invention
In order to solve the above problems, an object of the present application is to provide a method capable of performing image merging between multiple frame buffers corresponding to the use of a vertical synchronization technique based on a dirty rectangular region calculation.
Specifically, the invention provides a frame buffer merging method based on dirty rectangular region calculation, which comprises the following steps:
s1, after a frame buffer device driver applies for reserved frame buffering, setting the number of reserved frame buffering as MAX_NUM;
s2, for frame buffer synchronization of MAX_NUM, the following formula is satisfied:
Si=S(i-1)+Ri;
i=(i+1)%MAX_NUM;
Wherein i represents the index value corresponding to the frame buffer of the current operation;
Si represents the graphics data value of the ith frame buffer;
S (i-1) represents the graphics data value of the i-1 st frame buffer;
ri represents the graphic data currently required to be drawn;
after the synchronous action between the drawing of one graphic data and the frame buffering is finished, adding one bit of the current i value backwards, and performing the surplus action between the added value and MAX_NUM to ensure that the i value is in the frame buffering subscript range;
Step S3, when the frame buffer MAX_NUM=N, SN=S (N-1) +RN, and MAX_NUM=3 is set, i.e. 3 frame buffers are reserved, when the graphical user interface obtains the dirty rectangular region R1 to be drawn currently, R1 can be updated to the frame buffer S1, and i=1, i.e. 3 frame buffers are reserved
S1=R1;
S4, when the graphic user interface obtains a dirty rectangular region R2 to be drawn currently, R2 can be updated to a frame buffer S2, namely, the region after the R1 matrix and the R2 matrix are combined and de-duplicated is updated, and R1xR2 is used for representing the matrix combination and de-duplication, at the moment
S2=R1xR2;
S5, when the graphic user interface obtains the dirty rectangular region R3 to be drawn currently, the R3 can be updated to the frame buffer S3, at this time
S3=R1xR2xR3;
Step S6. When the graphical user interface has no new dirty rectangular area, i.e. the user is no longer drawing new images, no new dirty rectangular area is input at this time, for a frame buffer device with triple frame buffer vertical synchronization mechanism enabled, i.e. the case of MAX_NUM=3 and the case of MAX_NUM >1, it is still necessary to update the rectangular area in the record onto the frame buffer S1, where the selection of starting to update the frame buffer S1, the frame buffer S2, the frame buffer S3 is done in sequence, the frame buffer S1 is updated again after the update of the frame buffer S3, and the cycle is continued, the recorded rectangular area needs to be observed for a valid number of times not exceeding MAX_NUM, so that the R1 area is discarded from the record, since MAX_NUM is set to 3, there are only three frame buffers altogether, so that when one rectangular area, here, R1 area, is synchronized 3 times between frame buffers, all frame buffers have been updated, R1 area is no longer needed to be synchronized, and can be discarded, this is the case
S4=S1+R2xR3;
Step S7. For a frame buffer device that has a triple frame buffer vertical synchronization mechanism enabled without a new dirty rectangular region, i.e. the case of max_num=3, update on frame buffer S2 in order, at this time
S5=S2+R3;
Step S7 corresponds to the case where no new dirty rectangular region is input, and if there is a dirty rectangular region input, the process is normally performed according to step S4;
step S8. For a frame buffer device that has a triple frame buffer vertical synchronization mechanism enabled without a new dirty rectangular region, i.e. the case of max_num=3, update on frame buffer S3 in sequence, at this time
S6=S3;
Step S8 corresponds to the case where no new dirty rectangular region is input, and if there is a dirty rectangular region input, the process is normally performed in step S5.
In the step S1, the method for applying for reserving frame buffer by the frame buffer device driver reserves a frame buffer shared memory in the kernel display driver, the physical addresses are continuous, and the hardware is aligned, wherein the function of the reserved memory is a system function dma_alloc_pixel, and the parameter is the size of the memory.
In step S2, the frame buffer synchronization is to avoid waiting time of frame drawing by switching between multiple buffer areas in the graphics frame drawing process, so that data processing is smoother, at this time, the drawing process of the previous frame buffer needs to be copied during switching between multiple buffer areas, and data loss of drawing is avoided, and this process is the frame buffer synchronization.
The graphic user interface, that is, when the user needs to display the image on the frame buffer, the user designates the size and position of the displayed image and prepares the graphic data by the user, after the current dirty rectangular area to be drawn is obtained according to the size and position of the displayed image designated by the user, the width and height of the dirty rectangular area are obtained according to the size and position of the image, and the x and y values of the two-dimensional space of the image are obtained according to the display position of the image, wherein the origin of the corresponding coordinate system is the starting address of the frame buffer.
In the step S4, the specific steps of updating the region after merging and de-duplication of the R1 and R2 matrices are as follows:
Each rectangular structure body comprises four parameters, namely an abscissa x and an ordinate y of the upper left corner point, and a width w and a height h of the rectangle;
Firstly, judging whether two rectangles are intersected or not, wherein the judgment can be carried out by comparing the positions of the right boundary, the left boundary, the upper boundary and the lower boundary of the two rectangles;
If the right boundary of any rectangle is smaller than the left boundary of the other rectangle, or the left boundary is larger than the right boundary of the other rectangle, or the lower boundary is smaller than the upper boundary of the other rectangle, or the upper boundary is larger than the lower boundary of the other rectangle, the two rectangles are not intersected, and the original image before merging is directly returned;
if two rectangles intersect, the combined image can be represented by solving a union rectangle of the two rectangles, wherein the x coordinate of the left upper corner point of the new rectangle is the minimum value of the x coordinate of the left upper corner point of the two rectangles, the y coordinate of the left upper corner point is the minimum value of the y coordinate of the left upper corner point of the two rectangles, the width of the rectangle is the maximum value of the width of the two rectangles, and the height of the rectangle is the maximum value of the height of the two rectangles;
The right boundary of the rectangle after the duplication removal is the minimum value of the right boundary of the two rectangles, the left boundary is the maximum value of the left boundary of the two rectangles, the lower boundary is the minimum value of the lower boundary of the two rectangles, and the upper boundary is the maximum value of the upper boundary of the two rectangles;
and finally, outputting the rectangular output after the duplicate removal as a combined duplicate removal image.
The application has the advantages that by applying the scheme of the application, the requirement of multiple buffering on image merging among frame buffers can be met while using a dirty rectangle mechanism of the image user interface, and the image display effect of the frame buffer device is enhanced.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate and together with the description serve to explain the application.
Fig. 1 is a schematic diagram of an embodiment of the present method.
Fig. 2 is a schematic flow chart of the method.
Detailed Description
In order that the technical content and advantages of the present invention may be more clearly understood, a further detailed description of the present invention will now be made with reference to the accompanying drawings.
The application provides a frame buffer merging method based on dirty rectangular area calculation, and the method is shown in fig. 1, wherein the core method logic of the application is described as follows, S1 to S6 in the diagram refer to graphic memories operated by corresponding steps, R1, R2 and R3 refer to dirty rectangular areas input by a user, and represent a part of graphic areas which are required to be updated and correspond to S1, S2 and S3, and further as shown in fig. 2:
s1, after a frame buffer device driver applies for reserved frame buffer, the frame buffer device driver applies for reserved frame buffer by reserving a shared frame buffer memory in a kernel display driver, wherein physical addresses are continuous and hardware is aligned;
S2, for the frame buffer synchronization of MAX_NUM, the formula is required to be satisfied:
Si=S(i-1)+Ri;
i=(i+1)%MAX_NUM;
Wherein i represents the index value corresponding to the frame buffer of the current operation;
Si represents the graphics data value of the ith frame buffer;
S (i-1) represents the graphics data value of the i-1 st frame buffer;
ri represents the graphic data currently required to be drawn;
after the synchronous action between the drawing of one graphic data and the frame buffering is finished, adding one bit of the current i value backwards, and performing the surplus action between the added value and MAX_NUM to ensure that the i value is in the frame buffering subscript range;
The frame buffer synchronization is to avoid waiting time of frame drawing through switching among multiple buffer areas in the graphic frame drawing process, so that data processing is smoother, at the moment, the drawing process of the last frame buffer needs to be copied during switching among the multiple buffer areas, and drawing data loss is avoided, wherein the process is the frame buffer synchronization;
Step S3, when the frame buffer MAX_NUM=N, SN=S (N-1) +RN, setting MAX_NUM=3, namely reserving 3 frame buffers, when the graphic user interface, namely the user needs to display images on the frame buffer, the user can designate the size and the position of the displayed images, the graphic data are prepared by the user, after the dirty rectangular area R1 needing to be drawn currently is obtained according to the size and the position of the displayed images designated by the user, the width and the height of the dirty rectangular area are obtained according to the size and the position of the image, the x and the y values of the two-dimensional space of the image are obtained according to the display position of the image, the corresponding coordinate system origin, namely the starting address of the frame buffer, R1 can be updated to the frame buffer S1, and i=1 at the moment
S1=R1;
S4, when the graphic user interface obtains a dirty rectangular region R2 to be drawn currently, updating the dirty rectangular region R2 to a frame buffer S2, namely updating the region after the R1 matrix and the R2 matrix are combined and de-duplicated, and using R1xR2 to represent the matrix to combine and de-duplicated, wherein
S2=R1xR2;
The method comprises the following specific steps:
Each rectangular structure contains four parameters, an abscissa x, an ordinate y of the upper left corner point and a width w and a height h of the rectangle.
First, it is determined whether two rectangles intersect. The determination can be made by comparing the positions of the right, left, upper and lower boundaries of the two rectangles. If the right boundary of any rectangle is smaller than the left boundary of another rectangle, or the left boundary is larger than the right boundary of another rectangle, or the lower boundary is smaller than the upper boundary of another rectangle, or the upper boundary is larger than the lower boundary of another rectangle, the two rectangles are not intersected, and the original image before merging is directly returned.
If two rectangles intersect, the combined image can be represented by taking the union rectangle of the two rectangles. The x coordinate of the left upper corner of the new rectangle is the minimum value of the x coordinate of the left upper corner of the two rectangles, the y coordinate of the left upper corner is the minimum value of the y coordinate of the left upper corner of the two rectangles, the width of the rectangle is the maximum value of the width of the two rectangles, and the height of the rectangle is the maximum value of the height of the two rectangles.
Then, the overlapping area between the two rectangles needs to be removed. The de-duplicated rectangle can be calculated by judging the positions of the boundaries of the two rectangles. The right boundary of the rectangle after the duplication removal is the minimum value of the right boundary of the two rectangles, the left boundary is the maximum value of the left boundary of the two rectangles, the lower boundary is the minimum value of the lower boundary of the two rectangles, and the upper boundary is the maximum value of the upper boundary of the two rectangles.
And finally, outputting the rectangular output after the duplicate removal as a combined duplicate removal image.
S5, when the graphic user interface obtains the dirty rectangular region R3 to be drawn currently, the dirty rectangular region R3 can be updated to the frame buffer S3, and at the moment
S3=R1xR2xR3;
Step S6. When the graphical user interface has no new dirty rectangular area, i.e. the user is no longer drawing new images, no new dirty rectangular area is entered at this time, for the case of a frame buffer device with triple frame buffer vertical synchronization mechanism enabled, i.e. max_num=3 as mentioned above, and the case of max_num >1, it is still necessary to update the rectangular area in the record onto frame buffer S1, where the selection of starting to update frame buffer S1, frame buffer S2, frame buffer S3 is done in sequence, frame buffer S1 is updated again after updating frame buffer S3, and the cycle is continued, but the recorded rectangular area needs to be observed for a valid number of times not exceeding max_num, so the R1 area is discarded from the record, since max_num is set to 3, there are only three frame buffers in total, so when one rectangular area, e.g. R1 area, is synchronized 3 times between frame buffers, all frame buffers have been updated, R1 area is no longer needed to be synchronized, and R1 area is discarded, is no longer needed to be synchronized, as is 3
S4=S1+R2xR3;
Step S7 for the frame buffer device with triple frame buffer vertical synchronization mechanism enabled when no new dirty rectangular region is available, i.e. max_num=3, updating the frame buffer S2 in sequence, where step S7 corresponds to the case when no new dirty rectangular region is available, if there is a dirty rectangular region, the process is normally performed as in step S4, when
S5=S2+R3;
Step S8 for the case of a frame buffer device with triple frame buffer vertical synchronization mechanism enabled when no new dirty rectangular region is available, namely max_num=3, the update frame buffer S3 is sequentially performed, where step S8 corresponds to the case of no new dirty rectangular region input, if there is a dirty rectangular region input, the process is performed normally as in step S5, when
S6=S3。
In summary, the key point of the method of the present application is the relevant steps of the frame buffer image merging method.
The above description is only of the preferred embodiments of the present invention and is not intended to limit the present invention, and various modifications and variations can be made to the embodiments of the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.