KR102834455B1 - Memory management method - Google Patents

Abstract

A memory management method is disclosed. The method includes an atlas checking step of checking whether an image requested for rendering from an execution program is allocated to a texture atlas through a uv node list for a texture atlas, and a retained image response step of reading a uv node corresponding to the requested image from the uv node list and responding to the execution program if allocated. In addition, the uv node includes an image key uniquely assigned to the image, a memory address of the texture atlas, and uv coordinates which are allocation location information of the image within the texture atlas.

Description

{Memory management method} for image rendering

The present invention relates to three-dimensional computer graphics technology, and more particularly, to three-dimensional computer graphics technology using garbage collection.

In an environment that uses Garbage Collection (GC), when rendering multiple dynamic images in 3D computer graphics, a large amount of garbage memory is generated, which reduces rendering performance. Furthermore, in 3D graphics such as games or metaverses where real-time rendering is important, when rendering multiple dynamic images simultaneously, frequent pricking occurs, or in other words, the number of frames per second decreases, which causes problems that reduce usability.

Domestic Patent Publication No. 10-1703984 (announced on February 9, 2017)

The present invention aims to provide a technical solution that can prevent a decline in rendering performance even when rendering a large number of dynamic images.

A memory management method according to an aspect of the present invention may include an atlas checking step of checking whether an image requested for rendering from an execution program is allocated to a texture atlas through a uv node list for a texture atlas, and a retained image response step of reading a uv node corresponding to the requested image from the uv node list and responding to the execution program if so allocated. In addition, the uv node may include an image key uniquely assigned to the image, a memory address of the texture atlas, and uv coordinates which are allocation location information of the image within the texture atlas.

The memory management method may further include a step of responding to an unallocated image, which obtains the requested image from storage if the requested image is unallocated in the texture atlas, newly allocates the requested image to the texture atlas, and reads a uv node corresponding to the newly allocated image from the uv node list and responds to the executable program.

The non-owned image response step may include a step of inputting an image request into a queue, a step of processing the image request loaded into the queue to obtain the requested image from local storage or remote storage, a step of writing the obtained requested image into a read buffer, and a step of newly assigning the requested image written into the buffer to a texture atlas.

The present invention creates an effect of preventing a degradation of rendering performance even when rendering a large number of dynamic images in a 3D computer graphic environment using garbage collection (GC) through a proposed memory management method. Specifically, the present invention reduces GC activity by asynchronously loading a large number of images using a buffer and quickly packing them into a texture atlas, and improves usability by reducing CPU operations required for image rendering by quickly enabling a response to a requested image based on a texture atlas.

Figure 1 is a flowchart of a memory management method for image rendering according to one embodiment.
Figure 2 is a specific flow diagram of S300 according to one embodiment.
Figure 3 is a specific flow diagram of S340 according to one embodiment.
Figure 4 is a schematic diagram of a memory management system according to one embodiment.
FIG. 5 is a block diagram of a computing device according to one embodiment.

The above-described and additional aspects of the present invention will become more apparent through the preferred embodiments described with reference to the attached drawings. Hereinafter, the present invention will be described in detail through these embodiments so that those skilled in the art can easily understand and reproduce the present invention.

First, the terms used in this specification can be defined as follows.

■ Texture: A two-dimensional image applied to the surface of a three-dimensional object in 3D computer graphics.

■ Atlas: This refers to a two-dimensional image that combines multiple textures. The exact distinction in this specification is texture atlas.

■ Thread: refers to the smallest unit of execution flow of a computer program, and one process can have one or more threads.

■ Storage: refers to storage devices such as HDD and SDD on a computer.

■ Remote: Refers to an Internet environment where images can be downloaded, such as a file server, web server, or CDN.

■ Binary: This refers to the binary system consisting of only 0 and 1, and is the data format handled by computers.

■ Bitmap: One of the memory storage methods for saving images in the computer field, a method of storing the color number of each pixel in a two-dimensional array of pixels.

■ List data structure: A data structure that stores data by connecting nodes in a single line.

■ Node: refers to a data pointer and in this specification refers to the location information of an image assigned to the atlas.

■ uv: One of the coordinate systems used in computer graphics, it refers to two-dimensional coordinates for applying a two-dimensional image to the surface of a three-dimensional object.

FIGS. 1 to 3 are flowcharts for a memory management method for rendering a plurality of dynamic images in three-dimensional computer graphics in an environment using garbage collection, wherein FIG. 1 is a basic flowchart for a memory management method for image rendering according to one embodiment, FIG. 2 is a specific flowchart for S300 of FIG. 1 according to one embodiment, and FIG. 3 is a specific flowchart for S340 of FIG. 2 according to one embodiment. Before explanation, FIGS. 1 to 3 are performed by a computing device, and specifically, can be performed by one or more processors (processing units) which are hardware resources belonging to the computing device, and can be performed by a program (hereinafter referred to as an ‘atlas manager’) that is executed by the processor and processes an image request of a rendering target. Hereinafter, the performing entity of FIGS. 1 to 3 is referred to as an atlas manager.

Here, the description is made with reference to Fig. 1. The atlas manager receives a request for an image to be rendered from an executable program. Here, the executable program refers to a program that processes image rendering, such as a game program or a metaverse program, and requests the image by providing the atlas manager with an image key uniquely assigned to the image to be requested. The atlas manager verifies whether the image requested from the executable program exists in the texture atlas (S100). In the atlas verification step (S100), the atlas manager can check whether the image exists in the texture atlas by querying the uv node list for the texture atlas with the image key. Here, the uv node list is a list composed of uv nodes, which are information on textures (images) existing in the texture atlas, and the uv node format can be {image key, uv} or {image key, TA address, uv}, and the TA address means the memory address of the texture atlas that holds the image.

If the requested image exists in the texture atlas as a result of the verification at S100, the atlas manager reads the uv node corresponding to the requested image from the uv node list and responds to the execution program (S200). However, if the requested image is confirmed not to exist in the texture atlas, the atlas manager obtains the requested image and newly assigns it to the texture atlas (S300), and reads the uv node corresponding to the newly assigned image from the updated uv node list and responds to the execution program (S400). The former (S200) is called the possessed image response step, and the latter (S300, S400) is called the non-possessed image response step.

Hereinafter, a description will be given of Fig. 2. If the result of the verification in S100 shows that the requested image does not exist in the texture atlas, the atlas manager inputs an image request (image key) into the queue (S310). The atlas manager processes the image request loaded into the queue to obtain the requested image from a local storage or a remote storage (S320). At this time, if multiple image keys are loaded into the queue, multiple image requests can be processed simultaneously only by the number of threads. In other words, N image requests can be processed simultaneously as many as N threads. In one embodiment, the atlas manager first checks whether the requested image exists in the local storage with the image key in S320, and if so, reads and obtains the requested image from the local storage, and if the requested image does not exist in the local storage, downloads and obtains the requested image from the remote storage. Then, the downloaded image is stored in the local storage.

The atlas manager writes the image acquired from S320 to the buffer (S330). At this time, if the acquired image is a compressed image, the image can be decoded into a bitmap and then temporarily recorded in the buffer. Since the binary of the acquired image is generally compressed, such as JPG or PNG, it is decoded into a bitmap and written to the buffer. Next, the atlas manager newly assigns the image temporarily recorded in the buffer to the texture atlas (S340).

Hereinafter, the atlas manager will be described with reference to FIG. 3. In order to newly allocate an image recorded in a buffer to a texture atlas, the atlas manager checks the uv coordinates indicated by the first uv node of the uv node list (S340) and newly allocates an image to the confirmed uv coordinates (S342). Then, the first uv node is moved to the end of the uv node list to sort the list (S343). Meanwhile, in the held image response step (S200) of FIG. 1, the atlas manager can also update and manage the list by moving the uv node that responded to the image request to the end of the uv node list. As described above, the reason for moving the uv node of the requested image to the end of the uv node list is to ensure that uv nodes with high usage frequency are continuously maintained in the texture atlas and that images of uv nodes with low usage frequency are updated, thereby enabling the texture atlas to be managed with minimal operations.

FIG. 4 is a schematic diagram of a memory management system according to one embodiment. In FIG. 4, the main system (100) means an execution program that requests an image to be rendered, and the runtime atlas (230) in the atlas manager (200) means the texture atlas described above. The main system (100) requests the corresponding image by providing an image key to the atlas manager (200) (1). If the requested image exists in the uv node list, the atlas manager (200) aligns the corresponding uv node to the end of the uv node list, and responds the uv node for the request to the main system (100) (1-1). The uv node format is {image key, TA address, uv}. If the request details do not exist in the atlas (230), the atlas manager (200) places the request in a queue (210) (1-2).

The asynchronous downloader (220) processes N image requests simultaneously according to the limited number of threads (2). The asynchronous downloader (220) checks whether the requested image exists in the local storage (300). Specifically, if the requested image exists in the local storage (3-1), the asynchronous downloader (220) reads the image from the local storage (3-1), and if the requested image does not exist in the local storage (3-2), downloads it from the remote storage (3-2), and stores the image downloaded from the remote storage (3-2-1) in the local storage (3-2).

The atlas manager (200) writes the acquired image to the read buffer. If it is a compressed image, it is decoded into a bitmap and written to the read buffer (4). The atlas manager (200) assigns the image temporarily recorded in the read buffer to the uv coordinates indicated by the first uv node of the uv node list, and aligns the first uv node to the end of the uv node list (5). Next, the atlas manager (200) responds to the uv node for the image request to the main system (100).

FIG. 5 is a block diagram of a computing device according to one embodiment. FIG. 5 is an example diagram of a computing device configuration. The above-described memory management method may be performed by a computing device, and specifically, may be performed by one or more processors belonging to the computing device. As illustrated in FIG. 5, the computing device (500) may include a processor (510), a memory (520), a storage device (530), an input/output interface (540), and a communication interface (550). The processor (510) may include a microprocessor having one or more processing cores such as a central processing unit (CPU), a graphical processing unit (GPU), or the like. Data, etc. may be stored in the memory (520). The memory (520) may include one or more volatile and nonvolatile memories. The storage device (530) is a storage medium for storing data, commands, etc., and may include non-transitory storage. The storage device (530) may include at least some of a hard disk drive, flash memory, an optical disk, a magneto-optical disk, a universal serial bus (USB) drive, and the like.

The input/output interface (540) may include one or more input and/or output devices that allow a user to provide input and receive output, and to exchange data with other computing devices. The input/output interface (540) may be a mouse, a keypad or keyboard, a camera, an optical scanner, a network interface, a modem, or any other known I/O device, or may include combinations thereof. The input/output interface (540) may include at least some of a graphics engine, a display, one or more output drivers (e.g., a display driver), one or more audio speakers, and one or more audio drivers. The communications interface (550) may provide one or more interfaces for network communications (e.g., packet-based communications) between the computing device (500) and one or more other computing devices. For example, the communications interface (550) may include a network interface controller (NIC) or network adapter for communicating over an Ethernet or other wired network, or a Wireless NIC or wireless adapter for communicating over a wireless network, such as Wi-Fi.

A bus may include hardware, software, or both that interconnect components of a computing device. For example, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a front-side bus (FSB), a hypertransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an INFINIBAND interconnect, a low-pin-count (LPC) bus, a memory bus, a Micro Channel Architecture (MAC) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCIe) bus, a serial technology attachment (SATA) bus, a VESA Local (VL, VESA: Video Electronics Standard Association Local) bus, another suitable bus, or a combination thereof. The configurations of FIG. 5 are exemplary, and the computing device may include at least some of the configurations of FIG. 6, and may also include other configurations not shown.

Meanwhile, the above-described memory management method can be written as a computer program. The codes and/or code segments that constitute such a program can be easily inferred by a computer programmer in the relevant field. In addition, such a program can be stored in a recording medium that can be read by a computing device, and read and executed by the computing device, thereby implementing the above-described memory management method. Such a recording medium can be a magnetic recording medium, an optical recording medium, etc.

The present invention has been described with reference to preferred embodiments thereof. Those skilled in the art will appreciate that the present invention may be implemented in modified forms without departing from the essential characteristics of the present invention. Therefore, the disclosed embodiments should be considered in an illustrative rather than a restrictive sense. The scope of the present invention is set forth in the claims, not in the foregoing description, and all differences within the scope equivalent thereto should be construed as being included in the present invention.

Claims (10)

A memory management method for image rendering performed by one or more processors of a computing device,
An atlas verification step that checks whether an image requested for rendering from an executable program that handles image rendering is allocated to a texture atlas through the uv node list for the texture atlas;
A step of responding to an executable program by reading a uv node containing an image key corresponding to the requested image, a memory address of a texture atlas, and uv coordinates, which are allocation location information of the image within the texture atlas, from the uv node list, if allocated, and then moving the uv node to the end of the uv node list and sorting it after the response; and
An unallocated image response step of inputting an image request into a queue if unallocated, obtaining a requested image from a local storage or a remote storage according to the image request loaded into the queue and writing it to a read buffer, but if there are multiple image requests loaded into the queue, simultaneously processing multiple image requests limited by the number of threads and writing them to the read buffer, newly allocating the requested image written to the read buffer to a texture atlas, but newly allocating it to the uv coordinates according to the first uv node of the uv node list, moving the first uv node to the end of the uv node list and aligning it, and reading the uv node corresponding to the newly allocated requested image from the uv node list and responding to the executable program;
A memory management method for image rendering that includes . In paragraph 1,
The non-owned image response step is a memory management method for image rendering that reads and acquires the requested image from local storage if the requested image exists in local storage, downloads and acquires the requested image from remote storage if the requested image does not exist in local storage, and stores the image acquired by downloading in local storage. In paragraph 1,
The non-retained image response step is a memory management method for image rendering that decodes the acquired requested image into a bitmap and writes it to the read buffer if it is a compressed image. A program stored in a computer-readable recording medium for causing a computer to execute a method according to any one of claims 1 to 3. A computer-readable recording medium storing program instructions for executing a method according to any one of claims 1 to 3. delete delete delete delete delete