KR20240106637A - Electronic device and method for page fault handling - Google Patents

Abstract

According to an embodiment, an electronic device may obtain a page in which a fault occurs from a cache memory area. Provided is a page fault handling method comprising the steps of: determining whether to perform prefetching accompanied by a smart network interface card; applying a virtual memory offset record of a host accessed by a thread to a leaf algorithm to obtain a major offset; determining a first page batch to be prefetched by the host and a second page batch to be prefetched by the smart network interface card; transmitting a request batch to a remote memory server; and receiving, by the smart network interface card, the second page batch.

Description

Page fault handling device and method {ELECTRONIC DEVICE AND METHOD FOR PAGE FAULT HANDLING}

Hereinafter, technology regarding a page fault handling device and method is provided.

Existing network interface cards (NICs) were developed based on Application-Specific Integrated Circuit (ASIC) and have the advantage of being easy to mass produce and inexpensive, but it is impossible to change the internal logic. To solve this problem, a new type of smart network interface card (SmartNIC) was developed. SmartNIC can process various workloads on behalf of the host (HOST) by having its own onboard processor and the necessary computer resources.

As the demand for artificial intelligence grows, the amount of data that needs to be processed has increased. Accordingly, the demand for memory required by individual hosts also increased. Therefore, research has recently been conducted on memory separation techniques that can overcome the physical limitations of individual hosts. Their research mainly utilizes RDMA to use the memory of other nodes. However, these studies show high host CPU overhead and have the disadvantage of high page access latency.

The background technology described above is possessed or acquired by the inventor in the process of deriving the disclosure of the present application, and cannot necessarily be said to be known technology disclosed to the general public before this application.

An electronic device according to an embodiment includes a memory storing computer-executable instructions; a processor that accesses the memory and executes the instructions; host; and a smart network interface card (Smart NIC, Smart Network Interface Card), wherein the commands are provided with the smart network interface card based on metadata about a fault page of the host. Determine whether to perform prefetching, and based on the decision to perform prefetching, apply a record of the virtual memory offset of the host accessed by the thread to the leaf algorithm Obtain a major offset and, based on the major offset, determine a first page batch to be prefetched by the host and a second page batch to be prefetched by the smart network interface card; , transmitting a request batch based on the metadata for the determined first page layout and the metadata for the determined second page layout to a remote memory server, and the remote memory responding to the request. The host may receive the first page layout from a server and the smart network interface card may receive the second page layout.

The processor obtains a caching record of the fault page stored in the page information table of the host, and, based on the caching record, retrieves the cache memory region in which the fault page is stored from the host. and the hit information of the thread corresponding to the returned fault page can be updated in the host's thread information table.

The processor may decide to perform prefetching with the smart network interface card based on a case where metadata regarding the fault page does not exist in the cache information table of the host.

The processor obtains a caching record of the fault page stored in the page information table of the host, and, based on the caching record, retrieves the cache memory region in which the fault page is stored from the host. and the hit information of the thread corresponding to the returned fault page can be updated in the host's thread information table.

The processor may decide to perform prefetching with the smart network interface card based on a case where metadata regarding the fault page does not exist in the cache information table of the host.

The processor obtains, for each of a plurality of threads running in the host, offset records of a virtual memory included in the host for the plurality of threads, and selects each of the offset records of the plurality of threads by a majority vote. The major offset can be obtained by applying the leaf algorithm, which is an algorithm.

The processor selects pages to be prefetched based on the major offset and determines the first page placement among the selected pages based on an expected access time to the remote memory server, Among the selected pages, pages other than the determined first page arrangement may be determined as the second page arrangement.

The processor determines pages to be stored in the cache memory area of the host and pages to be stored in the cache memory area of the smart network interface card from among the determined second page arrangement, and provides metadata for the second page arrangement. can be delivered to the prefetcher of the smart network interface card through the RDMA (remote direct memory access) offloader of the host.

The processor transmits the request batch corresponding to the first page batch to a remote memory server through an RDMA executor of the host, and sends the request batch corresponding to the second page batch to the smart network. It can be transmitted to a remote memory server through the RDMA executor of the interface card.

Based on the interval of pages stored in the remote memory server, an expected latency time is obtained from a scheduler of the smart network interface card, and the expected latency time exceeds a predetermined threshold time. Based on the case, requests included in the request batch can be individually delivered to the remote memory server.

The processor measures the number of requests included in a sending queue of the smart network interface card, and based on the case where the measured number of requests exceeds a predetermined threshold, the first page batch or the The speed of reception of at least one batch of the second page batches can be changed.

The processor, based on when the second page batch is received from the remote memory server, transfers pages to be stored in the cache memory area of the host among the second page batch to the host, and provides thread information of the host. In the table, hit information of threads corresponding to the delivered pages can be updated.

The processor writes pages to be evict, among pages included in the received first page batch and the second page batch, in a write memory region of the host, and writes the pages to be evict to a write memory region of the host. A release request batch based on metadata for may be transmitted to the smart network interface card, and the smart network interface card may transmit pages corresponding to the release request batch to the remote memory server.

FIG. 1 is a diagram illustrating a page fault handling electronic device and a remote memory server according to an embodiment.
Figure 2 is a flowchart illustrating a page fault handling method according to an embodiment.
FIG. 3 is a diagram illustrating a host of a page fault handling electronic device and a smart network interface card of the electronic device, according to an embodiment.
FIG. 4 is a diagram illustrating a host of an electronic device and a smart network interface card of the electronic device in a cache hit situation according to an embodiment.
5 and 6 are diagrams illustrating a host of an electronic device in a cache miss situation and a smart network interface card of the electronic device, according to an embodiment.
FIG. 7 is a diagram illustrating network scheduling of a host of a page fault handling electronic device and a smart network interface card of the electronic device, according to an embodiment.
FIG. 8 is a diagram illustrating page fault handling of a host of an electronic device and page eviction of a smart network interface card of the electronic device, according to an embodiment.

Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only and may be changed and implemented in various forms. Accordingly, the actual implementation form is not limited to the specific disclosed embodiments, and the scope of the present specification includes changes, equivalents, or substitutes included in the technical idea described in the embodiments.

Terms such as first or second may be used to describe various components, but these terms should be interpreted only for the purpose of distinguishing one component from another component. For example, a first component may be named a second component, and similarly, the second component may also be named a first component.

When a component is referred to as being “connected” to another component, it should be understood that it may be directly connected or connected to the other component, but that other components may exist in between.

Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as “comprise” or “have” are intended to designate the presence of the described features, numbers, steps, operations, components, parts, or combinations thereof, and are intended to indicate the presence of one or more other features or numbers, It should be understood that this does not exclude in advance the possibility of the presence or addition of steps, operations, components, parts, or combinations thereof.

As used herein, “A or B”, “at least one of A and B”, “at least one of A or B”, “A, B or C”, “at least one of A, B and C”, and “A Each of phrases such as “at least one of , B, or C” may include any one of the items listed together in the corresponding phrase, or any possible combination thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the art. Terms as defined in commonly used dictionaries should be interpreted as having meanings consistent with the meanings they have in the context of the related technology, and unless clearly defined in this specification, should not be interpreted in an idealized or overly formal sense. No.

Hereinafter, embodiments will be described in detail with reference to the attached drawings. In the description with reference to the accompanying drawings, identical components will be assigned the same reference numerals regardless of the reference numerals, and overlapping descriptions thereof will be omitted.

FIG. 1 is a diagram illustrating a page fault handling electronic device and a remote memory server according to an embodiment.

The electronic device 100 according to one embodiment is a device for handling page faults, and includes a processor 110, a memory 120, a host 130, and a smart network interface. It may include a card (Smart NIC, Smart Network Interface Card).

The processor 110 may, for example, execute software to control at least one other component (e.g., hardware or software component) of the electronic device 100 connected to the processor 110, and process various data. Alternatively, calculations can be performed. According to one embodiment, as at least part of data processing or computation for page fault handling, processor 110 stores instructions or data in memory 120, and stores instructions or data stored in memory 120. can be processed, and the resulting data can be stored in the memory 120.

Memory 120 may include instructions executable by a computer. Memory 120 may temporarily and/or permanently store various data and/or information required for page fault handling. For example, the memory 120 may store at least one of page information, thread information, or metadata about page batch. Page information may include the virtual address, offset, type, and state of the page. Thread information may include the thread's ID, offset, and hit information. A page layout may be a batch containing at least two or more pages.

The host 130 may include a kernel area of the electronic device 100. For example, the host 130 may include the remaining areas excluding the user area among the memory space allocated to one process or at least one of a plurality of processes. The host 130 can be distinguished from the smart network interface card 140 as the CPU processes commands.

For example, the host 130 may prefetch a batch of pages including faulted pages. Specifically, the host 130 may receive a page layout including a faulted page from the remote memory server 150. The host 130 may prefetch the page batch received from the remote memory server 150. Prefetching is performed by the electronic device 100 to reduce the overhead generated when pages repeatedly move between a memory area (e.g., cache memory area) and a disk area (e.g., remote memory server 150). ) may represent a process of delaying the release time of pages in the cache memory area or pre-storing candidate pages (for example, pages in a page arrangement from which faulted pages are excluded). The host 130 may offload the above-described prefetching process to the smart network interface card 140. Host 130 may offload remote direct memory access (RDMA) to smart network interface card 140. Remote direct memory access (RDMA) performs at least one of requesting a page layout from the remote memory server 150 or receiving a page layout from the remote memory server 150 to prefetch a page layout including a fault page. It can be expressed.

The smart network interface card 140 may be a network interface card that can process various workloads on behalf of the host 130 by having its own on-board processor and computer resources necessary for the on-board processor. For example, the smart network interface card 140 may prefetch a batch of pages that include faulted pages. Specifically, the smart network interface card 140 may perform remote direct memory access (RDMA) on behalf of the host 130. The smart network interface card 140 can reduce the CPU utilization of the host 130 and reduce the page access latency of the host 130 by performing remote direct memory access (RDMA) on behalf of the host 130. You can do it.

The remote memory server 150 may be a server that is separate from the electronic device 100 and stores memory. Specifically, the remote memory server 150 may store a plurality of memories. The remote memory server 150 may transfer a plurality of memories to the electronic device 100 through at least one of the network interface card or the smart network interface card 140 of the electronic device 100.

Figure 2 is a flowchart illustrating a page fault handling method according to an embodiment.

In step 210, the electronic device (e.g., the electronic device 100 of FIG. 1), based on metadata about the fault page of the host (e.g., the host 130 of FIG. 1), It is possible to determine whether to perform prefetching with a network interface card (eg, smart network interface card 140 of FIG. 1). For example, metadata about a fault page may be structured data about information about the fault page. Metadata about the fault page may include at least one of the address of the fault page or the size of the fault page. Electronic devices can reduce latency caused by memory access through prefetching.

The electronic device according to one embodiment may determine whether to perform prefetching using a smart network interface card. For example, when a fault page is stored in a cache memory, the electronic device may return the fault page to the process from the location where the fault page is cached. In other words, when the fault page is stored in the cache memory, the electronic device may perform prefetching on the page arrangement including the fault page. Alternatively, the electronic device may decide to perform prefetching with a smart network interface card if the fault page is not stored in the cache memory. Below, in Figures 5 and 6, the prefetching process involving a smart network interface card is explained in detail.

In step 220, based on the decision to perform prefetching, the electronic device may obtain a major offset by applying the virtual memory offset record of the host accessed by the thread to the leaf algorithm. For example, the leaf algorithm uses a majority voting algorithm to obtain a major offset whenever a page fault occurs, based on the offset of the swap cache area stored in the record of the swap cache area offset of the faulted page. It can represent an algorithm that can be used. The swap cache area may be an area where pages are temporarily stored before they are evict to disk.

The electronic device according to one embodiment may record the virtual memory offset of the host accessed by the thread and obtain the major offset of the thread requesting the fault page. Here, the offset may be an integer representing the displacement difference from the swap cache area to the last page among pages to be prefetched based on the fault page. The electronic device may obtain the offset of each thread for each of the plurality of threads executing in the time interval in which the page fault occurred. The electronic device may obtain a major offset by applying the major offsets of a plurality of threads to a leaf algorithm. The electronic device can efficiently determine the arrangement of pages to be prefetched by obtaining a major offset through the offsets of a plurality of threads. For example, the electronic device can increase the cache hit ratio by determining the arrangement of pages to be prefetched through a major offset that reflects the virtual memory offset of the host accessed by the thread.

In step 230, the electronic device may determine a first page batch to be prefetched by the host and a second page batch to be prefetched by the smart network interface card, based on the major offset. For example, the electronic device may select pages to prefetch based on the major offset. The page to be prefetched may include a faulted page, and may include pages other than the faulted page in the swap cache area. The electronic device determines a major offset based on at least one of an expected access time to a remote memory server (e.g., remote memory server 150 of FIG. 1) or the urgency of the fault page. The first page arrangement may be determined among a plurality of pages selected as a basis. The electronic device may determine pages other than the determined first page layout among the selected pages as the second page layout. The urgency of a fault page may indicate the degree to which the fault page must be handled quickly by the host. The expected access time to the remote memory server may represent the expected time for at least one of the host of the electronic device or the smart network interface card to access the remote memory server. A detailed description of the expected access time for the remote memory server is described later in FIG. 7.

In step 240, the electronic device may transmit a request batch based on the metadata for the determined first page layout and the metadata for the determined second page layout to the remote memory server. For example, the electronic device may transmit a request batch corresponding to the first page batch to a remote memory server through an RDMA executor of the host. The electronic device may transmit the request batch corresponding to the second page batch to the remote memory server through the RDMA executor of the smart network interface card. Request placement may include information that the electronic device requests page placement from a remote memory server.

In step 250, the host may receive a first page batch and the smart network interface card may receive a second page batch from the remote memory server that responded to the request. The electronic device may deliver pages to be stored in the cache memory area of the host during the second page arrangement to the host, based on when the second page arrangement is received from the remote memory server. The electronic device may update the hit information of the thread corresponding to the delivered pages in the host's thread information table. Thereafter, the electronic device may perform prefetching based on the received page batches. The electronic device transmits the request batch of step 240 to the remote memory server, thereby reducing the overhead of the CPU by synchronously reading faulted pages and asynchronously reading pages to be prefetched. can be reduced.

FIG. 3 is a diagram illustrating a host of a page fault handling electronic device and a smart network interface card of the electronic device, according to an embodiment.

An electronic device (eg, the electronic device 100 of FIG. 1) according to an embodiment may include a host 300 and a smart network interface card 320.

The host 300 includes a cache information table 302, a thread information table 304, a cache memory region 306, and a write memory region. It may include (308), an RDMA offloader (310), and an RDMA executor (312).

The cache information table 302 may store metadata about pages that the host 300 prefetches and pages that it evicts. The thread information table 304 may include at least one of a virtual memory offset or hit metric of a host accessed by a thread. The cache memory area 306 stores batches for offloading request batches (e.g., request batch in FIG. 2) to the smart network interface card 320 and a remote memory server (e.g., remote memory server in FIG. 1). It may include the page layout received from (150)). The writing memory area 308 may include at least one page among pages written from the host 300 to the remote memory server or pages written from the smart network interface card 320 to the remote memory server. The RDMA offloader 310 may transmit a requested batch of pages to be prefetched to the smart network interface card 320. The RDMA executor 312 may perform remote direct memory access (RDMA) to a remote memory server.

The smart network interface card 320 may include a prefetcher 322, a scheduler 324, an RDMA executor 332, a cache memory area 326, and a write memory area 328.

The prefetcher 322 may determine pages to be stored in the cache memory area 306 of the host 300 and pages to be stored in the cache memory area 326 of the smart network interface card 320 from among the determined second page arrangement. there is. Additionally, the prefetcher 322 may receive a request batch of pages to be prefetched sent by the RDMA offloader 310 of the host 300. The scheduler 324 may dynamically schedule the process in which the request batch is delivered to the remote memory server. A description of the scheduler will be provided later with reference to FIG. 7. RDMA executor 332 may perform remote direct memory access (RDMA) to and from remote memory servers based on receiving a batch of requests from host 300. In other words, the RDMA executor 332 may receive page placement from a remote memory server on behalf of the host 300.

FIG. 4 is a diagram illustrating a host of an electronic device and a smart network interface card of the electronic device in a cache hit situation according to an embodiment.

An electronic device (e.g., the electronic device 100 of FIG. 1) according to an embodiment may obtain a page in which a fault has occurred from a cache memory area. The electronic device may prefetch the acquired fault page.

For example, the electronic device may obtain a caching record of the fault page stored in the cache information table 404 of the host 400 through metadata about the fault page. The caching record of the faulted page may include a record of whether the faulted page is a page stored in the cache memory area. The electronic device may return the fault page to the host 400 from the cache memory area where the fault page is stored, based on the above-described caching record.

Specifically, the cache memory area in which the fault page is stored may represent at least one memory area of the cache memory area 406 of the host 400 or the cache memory area 412 of the smart network interface card 410. If the fault page is stored in the cache memory area 412 of the smart network interface card 410, the electronic device returns the fault page to the host 400 from the cache memory area 412 by the RDMA offloader 408. You can. Thereafter, the electronic device may update the hit information (hit metric) of the thread regarding the fault page stored in the thread information table 402. The electronic device can record an offset for another prefetching by updating the hit information of the thread.

5 and 6 are diagrams illustrating a host of an electronic device in a cache miss situation and a smart network interface card of the electronic device, according to an embodiment.

An electronic device (e.g., the electronic device 100 of FIG. 1) according to an embodiment remotely directly connects to a remote memory server (e.g., the remote memory server 150 of FIG. 1) through a smart network interface card 610. Memory access (RDMA) can be performed. For example, the electronic device may receive a page arrangement (eg, a plurality of pages including a fault page) stored in a remote memory server through the smart network interface card 610. The electronic device may prefetch page batches received through the smart network interface card 610.

The electronic device according to one embodiment performs prefetching with the smart network interface card 610 based on the case where metadata related to the fault page does not exist in the cache information table 502 of the host 500. You can decide to do it. The electronic device may obtain a major offset by applying each offset record 504 of a plurality of threads running on the host 500 to the leaf algorithm, which is a majority voting algorithm.

The electronic device may determine the page arrangement 506 based on the major offset. Page batch 506 may include a first page batch to be prefetched by host 500 and a second page batch to be prefetched by a smart network interface card. Thereafter, from the determined second page arrangement, the electronic device may determine pages to be stored in the cache memory area of the host and pages to be stored in the cache memory area of the smart network interface card.

The electronic device may perform remote direct memory access (RDMA) on the first page batch including the faulted page through the RDMA executor 508. For example, the electronic device may transmit a request batch corresponding to the first page batch to a remote memory server through the RDMA executor 508 of the host 500. However, it is not limited to this, and the electronic device transmits a request batch based on metadata for the first page batch and the second page batch to the smart network interface card, so that the smart network interface card performs remote direct memory access (RDMA). It can be done.

The electronic device according to one embodiment may transmit a request layout based on metadata for the first page layout and/or the second page layout to the smart network interface card 610. For example, the electronic device may send metadata about the first page layout and/or the second page layout to the prefetcher 612 of the smart network interface card 610 through the RDMA offloader 604 of the host 600. It can be delivered to.

The electronic device may cause the smart network interface card 610 to perform remote direct memory access (RDMA) on behalf of the host 600 through the RDMA executor 614 of the smart network interface card 610. For example, the electronic device may transmit a request batch corresponding to the first page and/or second page batch to the remote memory server through the RDMA executor 614 of the smart network interface card 610.

Based on the case where the first page batch and/or the second page batch are received from the remote memory server, the electronic device selects the pages to be stored in the cache memory area 602 of the host 600 during the second page batch to the host ( 600). The electronic device may update the hit information of the thread corresponding to the delivered pages in the host's thread information table.

The electronic device can increase the cache hit ratio and reduce CPU overhead by allowing the smart network interface card 610 to perform remote direct memory access (RDMA) instead of the host 600. . For example, the electronic device allows a smart network interface card to perform remote direct memory access (RDMA), thereby reducing the number of instructions to be processed in the host 600 (e.g., instructions related to remote direct memory access). CPU overhead can be reduced. Additionally, the electronic device can increase the cache hit ratio through prefetching of more pages by creating a page arrangement including more pages.

FIG. 7 is a diagram illustrating network scheduling of a host of a page fault handling electronic device and a smart network interface card of the electronic device, according to an embodiment.

An electronic device (eg, the electronic device 100 of FIG. 1) according to an embodiment may perform network scheduling related to remote direct memory access (RDMA). For example, network scheduling for remote direct memory access (RDMA) involves an electronic device transmitting a request batch to a remote memory server (e.g., remote memory server 150 in FIG. 1) and a remote memory server responding to the request. This may be scheduling an operation in which an electronic device receives a page arrangement from a memory server.

The electronic device obtains the expected latency time from the scheduler (e.g., scheduler 324 in FIG. 3) of the smart network interface card 710 based on the interval of pages stored in the remote memory server. You can. The scheduler may schedule operations in which the electronic device transmits the requested batch to the remote memory server and receives the page batch from the remote memory server that responds to the request. The expected delay time may represent a time during which at least one of the host 700 or the smart network interface card 710 of the electronic device delays a response to a request from a remote memory server.

The electronic device may individually forward requests included in the request batch to the remote memory server based on a case where the expected delay time exceeds a predetermined threshold time. The electronic device can reduce the overhead caused by the remote direct memory access process and protect application traffic by dynamically forwarding the request batch to the remote memory server based on the expected delay time. can do. Additionally, the electronic device may determine the first page arrangement to be prefetched by the host 700 in the page arrangement, based on the expected access time.

The electronic device may cause the smart network interface card 710 to receive the first page batch and/or the second page batch from the remote memory server that responded to the request. The electronic device may change the speed of receiving a batch of pages based on a sending queue 718. For example, the electronic device may measure the number of requests included in the transmission queue 718 of the smart network interface card 710. The electronic device may change the speed of reception of the first page batch and/or the second page batch based on when the measured number of requests exceeds a predetermined threshold. For example, the electronic device may obtain an exponentially weighted moving average (EWMA) based on the measured number of requests. When the exponentially weighted average (EWMA) value exceeds the threshold, the electronic device may change the speed of requesting or receiving pages to be prefetched until the exponentially weighted average (EWMA) becomes less than the threshold.

Specifically, the smart network interface card 710 may include a page batch to be written 714 and a page batch to be read 716. The page batch to be written 714 may include pages stored in the writing memory area 712 of the smart network interface card 710. The batch of pages to be read 716 may include pages that will be prefetched by the smart network interface card 710 during the batch of pages 702 . The electronic device may delay a request for a batch of pages to be created 714 if the number of requests included in the transmission queue 718 exceeds a predetermined threshold. In other words, the electronic device reduces requests for a batch of pages to be written 714 until the number of requests contained in the transmission queue 718 is below a predetermined threshold, thereby reducing the number of requests for a batch of pages to be read 716. Request delays can be minimized. However, it is not limited to this, and the electronic device adjusts the number or size of pages to be prefetched based on the number of requests included in the transmission queue 718, thereby reducing the delay of the request for the batch of pages to be read 716. can be minimized.

FIG. 8 is a diagram illustrating page fault handling of a host of an electronic device and page eviction of a smart network interface card of the electronic device, according to an embodiment.

An electronic device (e.g., the electronic device 100 of FIG. 1) according to an embodiment may select pages to be evict among pages included in at least one of the received first page batch and the second page batch. Pages may be written to the write memory area 802 of the host 800. For example, the electronic device may write pages to be released in the writing memory area 802 of the host 800 and then update information on the pages to be released in the cache information table. The electronic device may generate a release request batch based on metadata about the pages to be released.

The electronic device may transmit the release request batch to the smart network interface card 810. For example, a batch of release requests may be delivered to the write memory area 812 of the smart network interface card 810 by the RDMA offloader 804 of the host 800. The electronic device may transmit a batch release request to the smart network interface card 810 based on when the pages stored in the writing memory area 802 exceed a predetermined threshold.

The electronic device may transfer pages corresponding to the release request batch to a remote memory server (eg, remote memory server 150 in FIG. 1) by the RDMA executor 814 of the smart network interface card 810. For example, the electronic device may evict pages to be emitted by delivering and writing pages corresponding to the ejection request batch to a remote memory server. The electronic device may delete information on pages to be released from the cache information table based on when the pages to be released have been released.

The embodiments described above may be implemented with hardware components, software components, and/or a combination of hardware components and software components. For example, the devices, methods, and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, and a field programmable gate (FPGA). It may be implemented using a general-purpose computer or a special-purpose computer, such as an array, programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and software applications running on the operating system. Additionally, a processing device may access, store, manipulate, process, and generate data in response to the execution of software. For ease of understanding, a single processing device may be described as being used; however, those skilled in the art will understand that a processing device includes multiple processing elements and/or multiple types of processing elements. It can be seen that it may include. For example, a processing device may include multiple processors or one processor and one controller. Additionally, other processing configurations, such as parallel processors, are possible.

Software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing unit to operate as desired, or may be processed independently or collectively. You can command the device. Software and/or data may be used on any type of machine, component, physical device, virtual equipment, computer storage medium or device to be interpreted by or to provide instructions or data to a processing device. , or may be permanently or temporarily embodied in a transmitted signal wave. Software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on a computer-readable recording medium.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. A computer-readable medium may include program instructions, data files, data structures, etc., singly or in combination, and the program instructions recorded on the medium may be specially designed and constructed for the embodiment or may be known and available to those skilled in the art of computer software. It may be possible. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -Includes magneto-optical media and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, etc. Examples of program instructions include machine language code, such as that produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or multiple software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described with limited drawings, those skilled in the art can apply various technical modifications and variations based on this. For example, the described techniques are performed in a different order than the described method, and/or components of the described system, structure, device, circuit, etc. are combined or combined in a different form than the described method, or other components are used. Alternatively, appropriate results may be achieved even if substituted or substituted by an equivalent.

Therefore, other implementations, other embodiments, and equivalents of the claims also fall within the scope of the claims described below.

Claims (23)

In a method of handling a page fault performed by an electronic device,
A step of determining whether to perform prefetching with a smart network interface card (Smart NIC) based on metadata about the fault page of the host. ;
Based on the decision to perform the prefetching, applying a record of the host's virtual memory offset accessed by a thread to a leaf algorithm to obtain a major offset;
based on the major offset, determining a first page batch to be prefetched by the host and a second page batch to be prefetched by the smart network interface card;
transmitting a request batch based on the metadata for the determined first page layout and the metadata for the determined second page layout to a remote memory server; and
The host receiving the first page batch and the smart network interface card receiving the second page batch from a remote memory server that responds to the request.
Including,
How to handle page faults.
According to paragraph 1,
The step of determining whether to perform the prefetching is:
Obtaining a caching record of the faulted page stored in a page information table of the host;
based on the caching record, returning the faulted page to the host from a cache memory region where the faulted page is stored; and
Updating hit information of a thread corresponding to the returned fault page in a thread information table of the host.
Including,
How to handle page faults.
According to paragraph 1,
The step of determining whether to perform the prefetching is:
determining to perform prefetching with the smart network interface card based on a case where metadata about the fault page does not exist in the cache information table of the host.
Including,
How to handle page faults.
According to paragraph 1,
The step of obtaining the major offset is,
For each of a plurality of threads running in the host, obtaining offset records of virtual memory included in the host for the plurality of threads; and
Obtaining the major offset by applying each offset record of the plurality of threads to the leaf algorithm, which is a majority voting algorithm.
Including,
How to handle page faults.
According to paragraph 1,
The step of determining the first page layout and the second page layout includes:
selecting pages to be prefetched based on the major offset;
determining the first page placement among the selected pages based on an expected access time to the remote memory server; and
Determining pages other than the determined first page arrangement among the selected pages as the second page arrangement.
Including,
How to handle page faults.
According to clause 5,
From the determined second page arrangement, determining pages to be stored in a cache memory area of the host and pages to be stored in a cache memory area of the smart network interface card; and
Passing metadata about the second page arrangement to a prefetcher of the smart network interface card through a remote direct memory access (RDMA) offloader of the host.
Containing more,
How to handle page faults.
According to paragraph 1,
The step of transmitting the request batch to the remote memory server is,
transmitting the request batch corresponding to the first page batch to a remote memory server through an RDMA executor of the host; and
Forwarding the request batch corresponding to the second page batch to a remote memory server through an RDMA executor of the smart network interface card.
Including,
How to handle page faults.
In clause 7,
Obtaining an expected latency time from a scheduler of the smart network interface card based on an interval of pages stored in the remote memory server; and
Individually forwarding requests included in the request batch to the remote memory server based on the case where the expected delay time exceeds a predetermined threshold time.
Containing more,
How to handle page faults.
According to paragraph 1,
Receiving the first page batch and the second page batch includes:
measuring the number of requests included in a sending queue of the smart network interface card; and
Changing the rate of receipt of at least one batch of the first page batch or the second batch of pages based on when the measured number of requests exceeds a predetermined threshold.
Including,
How to handle page faults.
According to paragraph 1,
Receiving the first page batch and the second page batch includes:
Based on when the second page batch is received from the remote memory server, delivering pages to be stored in a cache memory area of the host among the second page batch to the host; and
Updating hit information of threads corresponding to the delivered pages in the host's thread information table.
Including,
How to handle page faults.
According to paragraph 1,
writing pages to be evict, among pages included in the received first and second page batches, in a write memory region of the host;
transmitting to the smart network interface card a batch of release requests based on metadata for the pages to be released; and
The smart network interface card transmitting pages corresponding to the release request batch to the remote memory server.
Containing more,
How to handle page faults.
A computer program combined with hardware and stored in a computer-readable recording medium to execute the method of any one of claims 1 to 11.
In the page fault handling electronic device,
Memory storing computer-executable instructions;
a processor that accesses the memory and executes the instructions;
host; and
Smart NIC (Smart Network Interface Card)
Including,
The above commands are:
Based on metadata about a fault page of the host, determine whether to perform prefetching with the smart network interface card,
Based on the decision to perform the prefetching, a major offset is obtained by applying a record of the virtual memory offset of the host accessed by the thread to a leaf algorithm,
Based on the major offset, determine a first page batch to be prefetched by the host and a second page batch to be prefetched by the smart network interface card;
transmitting a request batch based on the metadata for the determined first page layout and the metadata for the determined second page layout to a remote memory server;
The host receives the first page batch from a remote memory server that responds to the request, and the smart network interface card receives the second page batch.
Electronic devices.
According to clause 13,
The processor,
Obtain a caching record of the faulted page stored in the page information table of the host,
Based on the caching record, the faulted page is returned to the host from a cache memory region where the faulted page is stored,
Updating the hit information of the thread corresponding to the returned fault page in the thread information table of the host.
Electronic devices.
According to clause 13,
The processor,
Based on the case where metadata about the fault page does not exist in the cache information table of the host, determining to perform prefetching with the smart network interface card
Electronic devices.
According to clause 13,
The processor,
For each of the plurality of threads running in the host, obtain offset records of virtual memory included in the host for the plurality of threads,
Obtaining the major offset by applying each offset record of the plurality of threads to the leaf algorithm, which is a majority voting algorithm.
Electronic devices.
According to clause 13,
The processor,
Select pages to be prefetched based on the major offset,
Based on an expected access time for the remote memory server, determine the first page placement among the selected pages,
Determining pages other than the determined first page arrangement among the selected pages as the second page arrangement.
Electronic devices.
According to clause 17,
The processor,
From the determined second page arrangement, determine pages to be stored in a cache memory area of the host and pages to be stored in a cache memory area of the smart network interface card,
Transferring metadata about the second page arrangement to a prefetcher of the smart network interface card through a remote direct memory access (RDMA) offloader of the host.
Electronic devices.
According to clause 13,
The processor,
Delivering the request batch corresponding to the first page batch to a remote memory server through an RDMA executor of the host,
transmitting the request batch corresponding to the second page batch to a remote memory server through an RDMA executor of the smart network interface card.
Electronic devices.
According to clause 19,
The processor,
Obtaining an expected latency time from a scheduler of the smart network interface card based on the interval of pages stored in the remote memory server,
Based on the case where the expected delay time exceeds a predetermined threshold time, the requests included in the request batch are individually delivered to the remote memory server.
Electronic devices.
According to clause 13,
The processor,
Measure the number of requests included in the sending queue of the smart network interface card,
changing the rate of receipt of at least one batch of the first page batch or the second batch of pages based on when the measured number of requests exceeds a predetermined threshold;
Electronic devices.
According to clause 13,
The processor,
Based on when the second page batch is received from the remote memory server, pages to be stored in the cache memory area of the host among the second page batch are delivered to the host,
Updating the hit information of the thread corresponding to the delivered pages in the host's thread information table.
Electronic devices.
According to clause 13,
The processor,
Among the pages included in the received first page batch and the second page batch, pages to be evict are written in a write memory region of the host,
send a batch of release requests based on metadata for the pages to be released to the smart network interface card;
The smart network interface card transmits pages corresponding to the release request batch to the remote memory server.
Electronic devices.

Images