CN109981674B - Remote procedure calling method, device, equipment and medium - Google Patents
Remote procedure calling method, device, equipment and medium Download PDFInfo
- Publication number
- CN109981674B CN109981674B CN201910271974.6A CN201910271974A CN109981674B CN 109981674 B CN109981674 B CN 109981674B CN 201910271974 A CN201910271974 A CN 201910271974A CN 109981674 B CN109981674 B CN 109981674B
- Authority
- CN
- China
- Prior art keywords
- data
- remote procedure
- message
- procedure call
- length
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/547—Remote procedure calls [RPC]; Web services
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/133—Protocols for remote procedure calls [RPC]
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/54—Indexing scheme relating to G06F9/54
- G06F2209/544—Remote
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/54—Indexing scheme relating to G06F9/54
- G06F2209/548—Queue
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Communication Control (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The application discloses remote procedure call device, including message sending module, message receiving module, management module, network module, coding and decoding module and code generation module, it is right through coding and decoding module remote procedure call request encodes, and carry out the burst when the length after remote procedure call request encoding is greater than preset length, every burst has the burst flag bit for whether this burst is the last burst in all bursts of remote procedure call message, then according to the burst to message sending module sends the data after the encoding, so that message sending module passes through network module to opposite terminal equipment sends the data after the encoding. Therefore, the client is prevented from occupying a large amount of system memory, the file is transmitted to the server as the process parameter through the fragmentation technology, and the transmission efficiency is improved through the transmission fragmentation. Corresponding methods, apparatus, and media are also disclosed.
Description
Technical Field
The present application relates to the field of communications, and in particular, to a method, an apparatus, a device, and a computer-readable storage medium for remote procedure call.
Background
With the development of communication technology, Remote Procedure Call (RPC) arises. By RPC, it is specifically meant requesting a service from a remote computer over a network without knowledge of the underlying network technology protocol.
RPC uses a client/server model, the requesting program acts as a client, and the service provider acts as a server. First, the client calling process sends a call message with process parameters (i.e., an RPC message) to the service process, and then waits for a response message. On the server side, the process remains in the sleep state until the RPC message arrives. When the RPC message arrives, the server obtains the process parameters, calculates the result based on the process parameters, sends reply information, waits for the next call information, and finally, the client calls the process to receive the reply information, obtains the process result, and then calls execution to continue.
Currently, in the implementation of RPC in Google Protocol Buffer of Google, when a client sends an RPC message to a service process, the RPC message is usually implemented by sending a TCP/UDP message. When the process parameter call information sent by the client, namely the data volume of the RPC message, is large, the client occupies a large amount of system memory, and even the message sending can not be completed because the memory is used up; and, the client cannot transmit the file as the process parameter call information to the server.
Disclosure of Invention
In view of this, the present application provides a remote procedure call method, which performs fragmentation processing on RPC messages when the RPC messages are large, so as to avoid a client from occupying a large amount of system memory, so that the memory usage is kept stable and controllable, and the file is transmitted to a server as a process parameter through a fragmentation technology. Correspondingly, the application also provides a remote procedure calling device, equipment, a computer readable storage medium and a computer program product.
A first aspect of the present application provides a remote procedure call device, the device comprising:
the system comprises a message sending module, a message receiving module, a management module, a network module, a coding and decoding module and a code generating module;
the network module is used for establishing connection with opposite terminal equipment and maintaining the connection;
the code generation module is used for generating a first code and a second code according to a remote procedure call message template, the first code is used for packaging user data conforming to the remote procedure call message template into a remote procedure call request, and the second code is used for analyzing the decoded remote procedure call message into the user data conforming to the remote procedure call message template;
the management module is used for sending the remote procedure call request to the message sending module;
the message sending module is used for packaging the remote procedure call request into a data structure to be stored in a message queue, taking out a first remote procedure call request from the head of the message queue when detecting that a message needs to be processed in the message queue, and sending the remote procedure call request to the coding and decoding module;
the encoding and decoding module is configured to encode the remote procedure call request, perform fragmentation when the length of the encoded remote procedure call request is greater than a preset length, where each fragment has a fragment flag bit and is used to represent whether the fragment is the last fragment of all fragments of a remote procedure call message, and then send encoded data to the message sending module according to the fragment, so that the message sending module sends the encoded data to the peer device through the network module;
the management module is further configured to register a message handler if a response of the peer device is required;
the message receiving module is used for monitoring the remote procedure call message, calling the coding and decoding module to decode when receiving the remote procedure call message, and sending the decoded message to the management module, so that the management module sends the decoded message to a corresponding message processor according to the serial number of the remote procedure call message.
Optionally, the encoding and decoding module is specifically configured to:
if the length of the remote procedure call request after coding is smaller than or equal to the preset length, sending coded data to the message sending module, and setting the fragment flag bit to represent that the current fragment is the last fragment of all fragments of the complete remote procedure call message;
if the length of the remote procedure call request after coding is larger than the preset length, when the coded length reaches the preset length, the coded data is sent to the message sending module, the fragment flag bit is set to represent that the current fragment is not the last fragment of all fragments of the complete remote procedure call message, then the coding step is continuously executed until the coded length is not larger than the preset length, the coded data is sent to the message sending module, and the fragment flag bit is set to represent that the current fragment is the last fragment of all fragments of the complete remote procedure call message.
Optionally, the encoding and decoding module is specifically configured to:
when the data structure is fixed-length data, the data structure is coded into a data type and original data;
and when the data structure is non-fixed-length data, the data structure is coded into a data type, a data length and original data.
Optionally, the encoding and decoding module applies in advance for a memory cache region for storing encoded data, and the size of the memory cache region is determined according to the number of bytes of the slice and the number of bytes of the maximum fixed-length data after encoding.
Optionally, the apparatus further comprises:
and the log module is used for recording the remote procedure call message between the local terminal equipment and the opposite terminal equipment.
A second aspect of the present application provides a remote procedure call method using the remote procedure call apparatus of the first aspect, the method comprising:
according to the user data which accords with the remote procedure call message template, calling a first code generated by a code generation module to generate a remote procedure call request;
packaging the remote procedure call request into a data structure to be stored in a message queue, and when detecting that a message needs to be processed in the message queue, taking out a first remote procedure call request from the head of the message queue and coding the remote procedure call request;
when the length of the remote procedure call request after coding is larger than the preset length, carrying out fragmentation, and sending coded data according to the fragmentation;
each fragment has a fragment flag bit for characterizing whether the fragment is the last fragment of all fragments of the remote procedure call message.
Optionally, when the length of the remote procedure call request after being encoded is less than or equal to a preset length, taking the encoded data as a fragment, and setting the fragment flag bit to represent that the current fragment is the last fragment of all fragments of the complete remote procedure call message;
when the length of the remote procedure call request after coding is greater than the preset length, when the coded length reaches the preset length, the coded data is sent, the fragment flag bit is set to represent that the current fragment is not the last fragment of all fragments of the complete remote procedure call message, then the coding step is continuously executed until the coded length is not greater than the preset length, the coded data is sent, and the fragment flag bit is set to represent that the current fragment is the last fragment of all fragments of the complete remote procedure call message.
Optionally, when the data structure is fixed-length data, the data structure is encoded into a data type and original data;
and when the data structure is non-fixed-length data, the data structure is coded into a data type, a data length and original data.
Optionally, the encoded data is stored in a memory buffer area, where the memory buffer area includes a data area and a to-be-processed area, the number of bytes in the data area is equal to the number of bytes in the fragment, and the number of bytes in the to-be-processed area is not less than the number of bytes after the maximum fixed-length data is encoded;
said encoding said remote procedure call request comprises:
if the data are stored in the area to be processed, the data in the area to be processed are moved to the data area;
if the coding is finished before the memory buffer area is used up, sending the coded data, and setting the zone bit of the fragment to represent that the current fragment is the last fragment of all fragments of the complete remote procedure call message;
if the coding is not completed when the memory buffer area is used up, sending coded data with specified length, wherein the specified length is equal to the number of bytes of the fragments, setting a fragment flag bit to represent that the current fragment is not the last fragment of all fragments of the complete remote procedure call message, and coding according to the following principle when the residual space of the data area is not enough to put down the last data type:
if the last data is of a fixed-length type, the data is encoded, and the content exceeding the data area is placed in the area to be processed;
when the last data is of a variable length type and the data length of the data can be accommodated in the data area, writing the data type and the data length of the data and the remaining data which can still be put into the data area, recording the currently coded data length, and continuously coding the remaining data when the next coding is carried out;
and when the last data is of a variable length type and the data area is not enough to accommodate the data type and the data length of the data, the current fragment does not encode the data into the data area any more.
Optionally, the method further includes:
if the response of the opposite terminal is needed, registering a message processor;
and monitoring the remote procedure call message, decoding the remote procedure call message when receiving the remote procedure call message, and sending the decoded message to a corresponding message processor according to the serial number of the remote procedure call message.
Optionally, the method further includes:
and recording the remote procedure call message between the local terminal and the opposite terminal.
A third aspect of an embodiment of the present application provides a remote procedure call device, where the device includes a processor and a memory:
the memory is used for storing a computer program;
the processor is configured to execute the remote procedure call method according to the first aspect of the present application.
A fourth aspect of embodiments of the present application provides a computer-readable storage medium for storing a computer program for executing the remote procedure call method according to the first aspect of the present application.
A fifth aspect of embodiments of the present application provides a computer program product comprising computer readable instructions which, when run on a computer, cause the computer to perform the remote procedure call method of the above-described aspects.
According to the technical scheme, the embodiment of the application has the following advantages:
the embodiment of the application provides a remote process calling device, which comprises a message sending module, a message receiving module, a management module, a network module, a coding and decoding module and a code generating module, wherein the coding and decoding module is used for coding a remote process calling request and carrying out fragmentation when the length of the coded remote process calling request is greater than a preset length, each fragmentation has a fragmentation flag bit and is used for representing whether the fragmentation is the last fragmentation of all the fragmentation of the remote process calling message or not, and then the coded data is sent to the message sending module according to the fragmentation so that the message sending module sends the coded data to an opposite terminal device through the network module; the management module is further configured to register a message handler if a response of the peer device is required; the message receiving module is used for monitoring the remote procedure call message, calling the coding and decoding module to decode when receiving the remote procedure call message, and sending the decoded message to the management module, so that the management module sends the decoded message to a corresponding message processor according to the serial number of the remote procedure call message.
The device carries out fragmentation processing on the message when the message is large and sends the message according to fragments, so that a client is prevented from occupying a large amount of system memory, the use of the memory is kept stable and controllable, and the file is transmitted to a server as a process parameter through a fragmentation technology. And after each fragment is sent out, the coding work of the next fragment can be immediately carried out without waiting for the completion of the coding of the whole message, thereby improving the message transmission efficiency.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without inventive exercise.
Fig. 1 is a schematic structural diagram of a remote procedure call device in an embodiment of the present application.
FIG. 2 is a flow chart of a remote procedure call method in an embodiment of the present application;
fig. 3 is a schematic structural diagram of a remote procedure call device in an embodiment of the present application.
Detailed Description
In order to make the technical solutions of the present application better understood, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The terms "first," "second," "third," "fourth," and the like in the description and in the claims of the present application and in the drawings described above, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the application described herein are, for example, capable of operation in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
When the network device acquires the network data, the network data needs to be uploaded to a server for analysis. When the network data is small, the data can be conveniently and orderly transmitted to the server through an RPC technology, such as Google Protocol Buffer of Google. However, when the amount of network data is large or a large file needs to be transmitted to a server, the common RPC technology cannot meet the requirement, and programmers often need to use other methods to transmit large blocks of data, for example, a module for transmitting a file is implemented by themselves to transmit a file specially, but the RPC technology cannot be used, so that the development efficiency is low and errors are prone to occur.
In view of this, the present application provides an RPC device, which can perform fragmentation processing on a message when the message is large, and send the message according to fragments, so as to avoid a client occupying a large amount of system memory, so that the memory usage is kept stable and controllable, and the transmission of a file serving as a process parameter to a server is achieved through a fragmentation technology. In addition, after each fragment is sent out, the coding work of the next fragment can be immediately carried out, and the whole message coding is not required to be finished, so that the message transmission efficiency is improved.
Next, the RPC device provided in the present application will be described with reference to specific embodiments.
Referring to fig. 1, a schematic diagram of a RPC device 100 includes: a network module 110, a code generation module 120, a management module 130, a message transmission module 140, a codec module 150, and a message reception module 160;
the network module 110 is configured to establish a connection with an opposite device and maintain the connection;
the code generating module 120 is configured to generate a first code and a second code according to a remote procedure call message template, where the first code is used to encapsulate user data conforming to the remote procedure call message template into a remote procedure call request, and the second code is used to parse the decoded remote procedure call message into user data conforming to the remote procedure call message template;
the management module 130 is configured to send the remote procedure call request to the message sending module;
the message sending module 140 is configured to encapsulate the remote procedure call request into a data structure, store the data structure in a message queue, and when detecting that there is a message in the message queue that needs to be processed, take out a first remote procedure call request from a head of the message queue and send the remote procedure call request to the codec module 150;
the encoding and decoding module 150 is configured to encode the remote procedure call request, and perform fragmentation when the length of the encoded remote procedure call request is greater than a preset length, where each fragment has a fragmentation flag bit, and is used to represent whether the fragment is the last fragment of all fragments of a remote procedure call message, and then send encoded data to the message sending module 140 according to the fragment, so that the message sending module 140 sends the encoded data to the peer device through the network module 110;
the management module 130 is further configured to register a message handler if a response of the peer device is required;
the message receiving module 160 is configured to monitor the remote procedure call message, call the encoding and decoding module 150 to decode when receiving the remote procedure call message, and send the decoded message to the management module 130, so that the management module 130 sends the decoded message to a corresponding message handler according to a sequence number of the remote procedure call message.
It is understood that the RPC apparatus 100 supports a variety of data types, including basic data types such as integer, Boolean, floating point, string, binary data, file data, where integers are divided into 1-byte, 2-byte, 4-byte or 8-byte integers from length, and signed and unsigned integers from whether they include symbols. It should be noted that the RPC device also supports complex data types such as classes, fixed-length arrays, variable-length arrays, and/or enumerations.
It should be noted that the data length of the fixed-length data type after encoding is also fixed, and the data length of the non-fixed-length data type after encoding is non-fixed.
In consideration of the decoding process, in practical applications, the encoding is often performed according to the following principle:
when the data structure is fixed-length data, the data structure is coded into a data type and original data;
and when the data structure is non-fixed-length data, the data structure is coded into a data type, a data length and original data.
As one example, fixed-length data is encoded as a 1-byte data type and raw data; the non-fixed-length data type is encoded as a 1-byte data type, a 4-byte data length, and original data.
In the technical scheme of the application, for the RPC message, a header describing the RPC message not only includes information such as a message size, a message sequence number, and a message type, but also includes a fragmentation flag bit for ensuring whether the current fragment is the last fragment of all fragments of a complete RPC message. Thus, when the opposite terminal receives any fragment, the opposite terminal determines whether the fragment is the last fragment of the complete RPC message through the fragment flag bit, if not, the opposite terminal continues to receive the next fragment and executes the step of determining whether the fragment is the last fragment, if so, the opposite terminal can determine the fragments between the last fragments received last two times, including the last fragment received last time, as all the fragments of the RPC message.
It is understood that the RPC device 100 supports the user to configure the size of each slice of the RPC message, i.e. the number of bytes of each slice, according to the actual requirement. In a specific implementation, the RPC fragment size may be configured by the management module 130. As an example, each slice may be configured to be 4096 bytes. Thus, when the RPC device 100 is used to transmit an RPC message, the RPC device 100 encodes an RPC request through the encoding and decoding module 150, specifically, the RPC device encodes an input parameter of a remote RPC request through the encoding and decoding module 150, if the size of encoded data exceeds 4096 bytes, 4096 bytes of data which is encoded currently will be sent to an opposite terminal, such as a server terminal, and then encoding is continued, and when the encoded data reaches 4096 bytes, the encoded data will be sent to the opposite terminal until all data are encoded and sent.
Based on this, in some possible implementations, the codec module 150 is specifically configured to: if the length of the RPC request after coding is less than or equal to the preset length, sending the coded data to the message sending module 140, and setting the fragment flag bit to represent that the current fragment is the last fragment of all fragments of the complete remote procedure call message; if the length of the RPC request after coding is less than or equal to the preset length, when the length after coding reaches the preset length, the coded data is sent to the message sending module 140, the fragment flag bit is set to represent that the current fragment is not the last fragment of all fragments of the complete remote procedure call message, then the coding step is continuously executed, when the length after coding is not more than the preset length, the coded data is sent to the message sending module 140, and the fragment flag bit is set to represent that the current fragment is the last fragment of all fragments of the complete remote procedure call message. Wherein, the preset length may be the length of the slice.
In a specific implementation, the codec module 150 may apply for a memory buffer area in advance for storing the encoded data, where the size of the memory buffer area is determined according to the number of bytes of the slice and the number of bytes of the maximum fixed-length data after encoding. Specifically, the memory cache region may include a data region and a to-be-processed region, where the size of the data region is equal to the size of the fragment, and the size of the to-be-processed region may be determined according to the number of bytes after the maximum fixed-length data is encoded, and based on this, the size of the memory cache region is not less than the sum of the number of bytes in the data region and the number of bytes after the maximum fixed-length data is encoded.
Based on this, when encoding the RPC request, the codec module 150 first checks whether the data exists in the region to be processed, and if so, moves the data in the region to be processed to the data region. The codec module 150 encodes the RPC request according to the data type of the RPC request, returns the encoded data to the message sending module 140 if encoding is completed before the memory buffer is exhausted, and informs that the current fragment is the last fragment of the RPC message through the fragment flag bit, and returns the data with the same size as the fragment in the encoded data as one fragment to the message sending module if encoding is still not completed when the memory buffer is exhausted, and informs that the current fragment is not the last fragment of the RPC message through the fragment flag bit.
When the data needs to be fragmented and the remaining space of the data area is not enough to put down the last data type, the data can be encoded according to the following principle:
when the last data is of a fixed-length type, encoding the data, and putting the content exceeding the data area into the area to be processed;
when the last data is of variable length type and the data area is enough to contain the data type and the data length of the data, writing the data type and the data length of the data and the remaining data which can still be put into the data area, recording the data length which is coded at the moment, and continuously coding the remaining data when the next coding is carried out;
when the last data has a variable length type and the data area does not contain the data type and the data length of the data, the current fragment does not encode the data into the data area.
In some possible implementations, the RPC apparatus 100 may further include a log module, configured to record RPC messages between the local device and the peer device, so as to be used when the user debugs.
By last, this application embodiment provides an RPC device, and the device includes message sending module, message receiving module, management module, network module, coding and decoding module and code generation module, and it is right through coding and decoding module remote procedure call request is encoded, and carry out the burst when length after the remote procedure call request is encoded is greater than preset length, and every burst has the burst marker bit for whether this burst of representation is last burst in all the bursts of remote procedure call message, then according to the burst to message sending module sends the data after the encoding, so that message sending module passes through network module to opposite terminal equipment sends data after the encoding.
According to the method, when the message is large, the message is subjected to fragmentation processing and is sent according to fragments, so that a client is prevented from occupying a large amount of system memory, the use of the system memory is kept stable and controllable, and the file is transmitted to a server as a process parameter through a fragmentation technology. And after each fragment is sent out, the coding work of the next fragment can be immediately carried out without waiting for the completion of the coding of the whole message, thereby improving the message transmission efficiency.
Based on the RPC device provided in the embodiment of the present application, an RPC method is further provided in the embodiment of the present application, which specifically includes RPC message sending and RPC message receiving.
Referring to the flow chart of the RPC method shown in fig. 2, the method includes:
s201: and calling the first code generated by the code generation module to generate a remote procedure calling request according to the user data conforming to the remote procedure calling message template.
The RPC message template refers to a standardized format of the RPC message, and defines fields included in the RPC message, lengths of the fields, and the like. In actual application, the RPC message template may be user-defined or pre-built, which is not limited in this embodiment.
After the user data which accords with the RPC message template is generated, the home terminal, namely the client terminal, calls the first code generated by the code generation module, and generates the RPC request according to the user data.
S202: and packaging the remote procedure call request into a data structure to be stored in a message queue, and when detecting that a message in the message queue needs to be processed, taking out a first remote procedure call request from the head of the message queue and coding the remote procedure call request.
The client encapsulates the RPC request into a data structure through the message sending module and stores the data structure in the message queue, when a message is detected to be processed in the message queue, namely the data structure is stored in the message queue, the first RPC request is taken out from the head of the message queue, specifically, the data structure corresponding to the first RPC request is taken out, and the RPC request is encoded through the encoding and decoding module.
S203: and when the length of the remote procedure call request after coding is larger than the preset length, carrying out fragmentation, and sending the coded data according to the fragmentation.
Each fragment has a fragment flag bit for characterizing whether the fragment is the last fragment of all fragments of the remote procedure call message. Therefore, after receiving the fragments, the opposite end can determine whether the fragments are the last fragments of all fragments of the RPC message according to the fragment flag bit, and further determine all fragments corresponding to the RPC message.
In practical application, when the length of the RPC request after coding is smaller than or equal to the preset length, the coded data is used as one fragment, and the fragment flag bit is set to represent that the current fragment is the last fragment of all fragments of the complete remote procedure call message, so that RPC message transmission is completed through one fragment.
When the length of the RPC request after coding is larger than the preset length, when the length of the RPC request after coding reaches the preset length, the coded data is sent, the fragment flag bit is set to represent that the current fragment is not the last fragment of all fragments of the complete remote procedure call message, then the coding step is continuously executed until the length of the coded fragment is not larger than the preset length, the coded data is sent, and the fragment flag bit is set to represent that the current fragment is the last fragment of all fragments of the complete remote procedure call message.
It should be noted that when the data structure is fixed-length data, it may be encoded into a data type and original data, and when the data structure is non-fixed-length data, it may be encoded into a data type, a data length and original data.
In some possible implementation manners, a memory cache region may also be applied in advance for storing the encoded data. Specifically, the memory cache region may include a data region and a to-be-processed region, where the number of bytes in the data region is equal to the number of bytes in the segment, and the number of bytes in the to-be-processed region is not less than the number of bytes after the maximum fixed-length data is encoded.
Based on the above, the application also provides an implementation manner for encoding the RPC request, and specifically, if the to-be-processed area stores data, the data of the to-be-processed area is moved to the data area; if the coding is finished before the memory buffer area is used up, sending the coded data, and setting the zone bit of the fragment to represent that the current fragment is the last fragment of all fragments of the complete remote procedure call message; if the coding is not completed when the memory buffer is used up, sending coded data with specified length, wherein the specified length is equal to the number of bytes of the fragments, and setting the fragment flag bit to represent that the current fragment is not the last fragment of all fragments of the complete remote procedure call message.
And, when the remaining space of the data area is not enough to put down the last data type, encoding according to the following principle:
if the last data is of a fixed-length type, the data is encoded, and the content exceeding the data area is placed in the area to be processed;
when the last data is of a variable length type and the data length of the data can be accommodated in the data area, writing the data type and the data length of the data and the remaining data which can still be put into the data area, recording the currently coded data length, and continuously coding the remaining data when the next coding is carried out;
and when the last data is of a variable length type and the data area is not enough to accommodate the data type and the data length of the data, the current fragment does not encode the data into the data area any more.
In the above specific implementation manner of the RPC method provided from the message sending perspective in the embodiment of the present application, in actual application, the local device may also be called remotely, that is, the local device may also serve as a server, at this time, the local device may receive an RPC message sent by an opposite device, and a message receiving process will be described in detail below.
In some possible implementation manners, if the local device needs to respond to the opposite terminal, the local device may register a message handler in the management module, then monitor the RPC message, decode the RPC message when receiving the RPC message, and send the decoded message to the corresponding message handler according to the sequence number of the RPC message.
In actual application, the local terminal device may further record a remote procedure call message between the local terminal and the opposite terminal, so as to be used by a user during debugging.
The embodiment of the application provides an RPC method, and when the RPC message is transmitted, the RPC message is divided into slices according to the size of the RPC message and transmitted, so that a client is prevented from occupying a large amount of system memory, the memory usage is kept stable and controllable, and a file is transmitted to a server as a process parameter through a fragmentation technology. And after each fragment is sent out, the coding work of the next fragment can be immediately carried out without waiting for the completion of the coding of the whole message, thereby improving the message transmission efficiency.
On the other hand, the method also operates that the user sets the fragment size of the RPC message by himself, so that the method is adaptive to the system of the user to achieve the best transmission efficiency. For the data of the file type, the data in the file is read into the memory when the data is coded, so that the transmission of a large file can be supported. The method also provides for status encoding and decoding, when an RPC message is fragmented, to record the location of the current message encoding or decoding so that processing of the next fragment can continue based on that location.
Based on the above method and apparatus provided by the embodiment of the present application, the present application further provides an RPC device, see the schematic structural diagram of the RPC device shown in fig. 3, where the RPC device includes a memory 301 and a processor 302, where:
the memory 301 is used for storing a computer program;
the processor 302 is configured to execute the RPC method provided in the embodiment of the present application according to the computer program.
The embodiment of the application also provides a computer readable storage medium, which is used for storing a computer program, wherein the computer program is used for executing the RPC method.
Embodiments of the present application also provide a computer program product comprising computer readable instructions, which when run on a computer, cause the computer to perform the remote procedure call method of the above aspects.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
It should be understood that in the present application, "at least one" means one or more, "a plurality" means two or more. "and/or" for describing an association relationship of associated objects, indicating that there may be three relationships, e.g., "a and/or B" may indicate: only A, only B and both A and B are present, wherein A and B may be singular or plural. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. "at least one of the following" or similar expressions refer to any combination of these items, including any combination of single item(s) or plural items. For example, at least one (one) of a, b, or c, may represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", wherein a, b, c may be single or plural.
The above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions in the embodiments of the present application.
Claims (13)
1. An apparatus for remote procedure call, the apparatus comprising:
the system comprises a message sending module, a message receiving module, a management module, a network module, a coding and decoding module and a code generating module;
the network module is used for establishing connection with opposite terminal equipment and maintaining the connection;
the code generation module is used for generating a first code and a second code according to a remote procedure call message template, the first code is used for packaging user data conforming to the remote procedure call message template into a remote procedure call request, and the second code is used for analyzing the decoded remote procedure call message into the user data conforming to the remote procedure call message template;
the management module is used for sending the remote procedure call request to the message sending module;
the message sending module is used for packaging the remote procedure call request into a data structure to be stored in a message queue, taking out a first remote procedure call request from the head of the message queue when detecting that a message needs to be processed in the message queue, and sending the remote procedure call request to the coding and decoding module;
the encoding and decoding module is configured to encode the remote procedure call request, perform fragmentation when the length of the encoded remote procedure call request is greater than a preset length, where each fragment has a fragment flag bit and is used to represent whether the fragment is the last fragment of all fragments of a remote procedure call message, and then send encoded data to the message sending module according to the fragment, so that the message sending module sends the encoded data to the peer device through the network module;
the management module is further configured to register a message handler if a response of the peer device is required;
the message receiving module is used for monitoring the remote procedure call message, calling the coding and decoding module to decode when receiving the remote procedure call message, and sending the decoded message to the management module, so that the management module sends the decoded message to a corresponding message processor according to the serial number of the remote procedure call message.
2. The remote procedure call device according to claim 1, wherein the codec module is specifically configured to:
if the length of the remote procedure call request after coding is larger than the preset length, when the coded length reaches the preset length, the coded data is sent to the message sending module, the fragment flag bit is set to represent that the current fragment is not the last fragment of all fragments of the complete remote procedure call message, then the coding step is continuously executed until the coded length is not larger than the preset length, the coded data is sent to the message sending module, and the fragment flag bit is set to represent that the current fragment is the last fragment of all fragments of the complete remote procedure call message.
3. The remote procedure call device according to claim 1, wherein the codec module is specifically configured to:
when the data structure is fixed-length data, the data structure is coded into a data type and original data;
and when the data structure is non-fixed-length data, the data structure is coded into a data type, a data length and original data.
4. The remote procedure call device according to claim 1, wherein the codec module applies for a memory buffer in advance for storing the encoded data, and the size of the memory buffer is determined according to the number of bytes of the slice and the number of bytes of the maximum fixed-length data after encoding.
5. The apparatus of any one of claims 1 to 4, further comprising:
and the log module is used for recording the remote procedure call message between the local terminal equipment and the opposite terminal equipment.
6. A remote procedure call method applied to the remote procedure call apparatus according to any one of claims 1 to 5, the method comprising:
according to the user data which accords with the remote procedure call message template, calling a first code generated by a code generation module to generate a remote procedure call request;
packaging the remote procedure call request into a data structure to be stored in a message queue, and when detecting that a message needs to be processed in the message queue, taking out a first remote procedure call request from the head of the message queue and coding the remote procedure call request;
when the length of the remote procedure call request after coding is larger than the preset length, carrying out fragmentation, and sending coded data according to the fragmentation;
each fragment has a fragment flag bit for characterizing whether the fragment is the last fragment of all fragments of the remote procedure call message.
7. The method of claim 6, wherein when the encoded length of the remote procedure call request is greater than a preset length, the encoded data is sent when the encoded length reaches the preset length, and the fragment flag is set to indicate that the current fragment is not the last of all fragments of the complete remote procedure call message, and then the encoding step is continued until the encoded length is not greater than the preset length, the encoded data is sent, and the fragment flag is set to indicate that the current fragment is the last of all fragments of the complete remote procedure call message.
8. The method of claim 6, wherein the data structure is fixed-length data, and is encoded as a data type and original data;
and when the data structure is non-fixed-length data, the data structure is coded into a data type, a data length and original data.
9. The method according to claim 6, wherein the encoded data is stored in a memory buffer, the memory buffer comprises a data area and a to-be-processed area, the number of bytes in the data area is equal to the number of bytes in the slice, and the number of bytes in the to-be-processed area is not less than the maximum number of bytes after the fixed-length data is encoded;
said encoding said remote procedure call request comprises:
if the data are stored in the area to be processed, the data in the area to be processed are moved to the data area;
if the coding is finished before the memory buffer area is used up, sending the coded data, and setting the zone bit of the fragment to represent that the current fragment is the last fragment of all fragments of the complete remote procedure call message;
if the coding is not completed when the memory buffer area is used up, sending coded data with specified length, wherein the specified length is equal to the number of bytes of the fragments, setting a fragment flag bit to represent that the current fragment is not the last fragment of all fragments of the complete remote procedure call message, and coding according to the following principle when the residual space of the data area is not enough to put down the last data type:
if the last data is of a fixed-length type, the data is encoded, and the content exceeding the data area is placed in the area to be processed;
when the last data is of a variable length type and the data length of the data can be accommodated in the data area, writing the data type and the data length of the data and the remaining data which can still be put into the data area, recording the currently coded data length, and continuously coding the remaining data when the next coding is carried out;
and when the last data is of a variable length type and the data area is not enough to accommodate the data type and the data length of the data, the current fragment does not encode the data into the data area any more.
10. The method according to any one of claims 6 to 9, further comprising:
if the response of the opposite terminal is needed, registering a message processor;
and monitoring the remote procedure call message, decoding the remote procedure call message when receiving the remote procedure call message, and sending the decoded message to a corresponding message processor according to the serial number of the remote procedure call message.
11. The method according to any one of claims 6 to 9, further comprising:
and recording the remote procedure call message between the local terminal and the opposite terminal.
12. A remote procedure call device, the device comprising a processor and a memory:
the memory is used for storing a computer program;
the processor is configured to execute the remote procedure call method of any of claims 6 to 11 in accordance with the computer program.
13. A computer-readable storage medium for storing a computer program for executing the remote procedure call method of any one of claims 6 to 11.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910271974.6A CN109981674B (en) | 2019-04-04 | 2019-04-04 | Remote procedure calling method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910271974.6A CN109981674B (en) | 2019-04-04 | 2019-04-04 | Remote procedure calling method, device, equipment and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109981674A CN109981674A (en) | 2019-07-05 |
| CN109981674B true CN109981674B (en) | 2021-08-17 |
Family
ID=67083058
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910271974.6A Active CN109981674B (en) | 2019-04-04 | 2019-04-04 | Remote procedure calling method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109981674B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111147573A (en) | 2019-12-24 | 2020-05-12 | 网宿科技股份有限公司 | A method and device for data transmission |
| CN115801713A (en) * | 2022-09-20 | 2023-03-14 | 杭州和利时自动化有限公司 | Data transmission method and device and readable storage medium |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1852209A (en) * | 2006-05-31 | 2006-10-25 | 北京和利时系统工程股份有限公司 | Remote process transfering method and system in distribution type control system |
| CN102546612A (en) * | 2011-12-23 | 2012-07-04 | 华中科技大学 | Remote procedure call implementation method based on remote direct memory access (RDMA) protocol in user mode |
| CN102594891A (en) * | 2012-02-17 | 2012-07-18 | 中国科学院计算技术研究所 | Method and system for processing remote procedure call request |
| CN103440244A (en) * | 2013-07-12 | 2013-12-11 | 广东电子工业研究院有限公司 | A big data storage optimization method |
| US9491261B1 (en) * | 2013-07-29 | 2016-11-08 | Amazon Technologies, Inc. | Remote messaging protocol |
| CN106453250A (en) * | 2016-09-05 | 2017-02-22 | 掌阅科技股份有限公司 | Processing method of big data RPC (Remote Procedure Call Protocol) |
| CN106534249A (en) * | 2016-09-21 | 2017-03-22 | 苏州市广播电视总台 | File transmission system based on file straight-through technology |
| CN106648940A (en) * | 2017-03-13 | 2017-05-10 | 北京百悟科技有限公司 | Remote procedure call method and device |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7290267B2 (en) * | 2002-01-23 | 2007-10-30 | International Business Machines Corporation | Multi-protocol object distribution |
-
2019
- 2019-04-04 CN CN201910271974.6A patent/CN109981674B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1852209A (en) * | 2006-05-31 | 2006-10-25 | 北京和利时系统工程股份有限公司 | Remote process transfering method and system in distribution type control system |
| CN102546612A (en) * | 2011-12-23 | 2012-07-04 | 华中科技大学 | Remote procedure call implementation method based on remote direct memory access (RDMA) protocol in user mode |
| CN102594891A (en) * | 2012-02-17 | 2012-07-18 | 中国科学院计算技术研究所 | Method and system for processing remote procedure call request |
| CN103440244A (en) * | 2013-07-12 | 2013-12-11 | 广东电子工业研究院有限公司 | A big data storage optimization method |
| US9491261B1 (en) * | 2013-07-29 | 2016-11-08 | Amazon Technologies, Inc. | Remote messaging protocol |
| CN106453250A (en) * | 2016-09-05 | 2017-02-22 | 掌阅科技股份有限公司 | Processing method of big data RPC (Remote Procedure Call Protocol) |
| CN106534249A (en) * | 2016-09-21 | 2017-03-22 | 苏州市广播电视总台 | File transmission system based on file straight-through technology |
| CN106648940A (en) * | 2017-03-13 | 2017-05-10 | 北京百悟科技有限公司 | Remote procedure call method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109981674A (en) | 2019-07-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10817460B2 (en) | RDMA data sending and receiving methods, electronic device, and readable storage medium | |
| JP6905066B2 (en) | Coding and decoding methods and devices | |
| CN112333186B (en) | Data communication method, device, equipment and storage medium | |
| CN112685248A (en) | Intelligent network card monitoring log obtaining method and device, electronic equipment and storage medium | |
| CN112911024B (en) | Method and system for collecting data of Internet of things, electronic device and storage medium | |
| CN109981674B (en) | Remote procedure calling method, device, equipment and medium | |
| CN103516788A (en) | Data pushing method and Flash client terminal and server thereof | |
| CN108460044B (en) | Data processing method and device | |
| JP7282895B2 (en) | Data retransmission decoding method, device, system and communication device | |
| CN109995589B (en) | Log collection method and system | |
| CN106209942B (en) | Data compression transmission method and system, terminal and server thereof | |
| CN112188562B (en) | Multicast scheduling method and device for relay base station, storage medium and electronic device | |
| CN110704361A (en) | RDMA (remote direct memory Access) data transmitting and receiving method, electronic equipment and readable storage medium | |
| CN109379083A (en) | Base64 code conversion method and electronic equipment based on educational system | |
| CN105262491A (en) | Data compression method, data compression system, data decompression method, data decompression system and communication equipment | |
| CN104539506A (en) | Network transmission service component and network transmission system | |
| CN113507530B (en) | Data forwarding method, related system and device and storage medium | |
| CN106878054B (en) | Service processing method and device | |
| CN111064976A (en) | Method for sending live broadcast information and server | |
| US10742783B2 (en) | Data transmitting apparatus, data receiving apparatus and method thereof having encoding or decoding functionalities | |
| CN107294878A (en) | Data transmission method, system and relevant device | |
| CN101699914A (en) | Up-link and down-link service data processing method and relevant device in mobile terminal | |
| CN112597082B (en) | Bus data transmission method and electronic equipment | |
| CN116893987B (en) | Hardware acceleration method, hardware accelerator and hardware acceleration system | |
| KR20060057008A (en) | How to monitor communications and systems and monitoring protocols |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |