CN117955648B - A key negotiation system and method based on DPDK architecture - Google Patents

Abstract

The invention relates to the technical field of network security, in particular to a key negotiation system and method based on a DPDK architecture. The invention redesigns the functions of key management algorithm, SAD, SPD structure and stored design, IKE configuration, etc., the above functions are realized by a key negotiation module, the functions of network data receiving and transmitting, service data encrypting and decrypting, data packaging/decapsulating, negotiation flow triggering, etc. are processed by a service processing module, and the management configuration module issues the contents of system configuration, security policy, etc. to the service processing module and the key negotiation module. The invention solves the defect that the open source framework of the traditional key agreement is excessively dependent on CPU performance due to the fact that the open source framework cannot be used based on a DPDK framework, ensures the synchronism and stability interaction among all modules in the equipment, and ensures that the VPN equipment has higher reliability.

Description

A key negotiation system and method based on DPDK architecture

Technical Field

The present invention relates to the field of network security technology, and in particular to a key negotiation method and system based on a DPDK architecture.

Background technique

With the development of computer network security, network key exchange technology has emerged. Through network key exchange technology, encrypted communication can be carried out between different VPN (Virtual Private Network) gateways. In traditional technology, IKE (Internet Key Exchange) negotiation is carried out between different VPN gateways to negotiate the keys of both parties, so as to realize encryption and decryption communication between the two gateways through the keys.

In the new network situation, facing a large number of high-throughput and high-concurrency network environments, the network data collection mechanism based on the traditional operating system kernel has been unable to match the existing needs in terms of packet loss rate and throughput. In the current high-speed network, high-performance systems need to successfully collect and process a large amount of data in an extremely limited time. Therefore, how to efficiently, completely and quickly capture data packets is the basis for accurate analysis of network data and the key to the next step of management and control.

DPDK is a collection of libraries and drivers designed for fast processing of network packets. DPDK replaces the kernel system call through the data plane function of the environment abstraction layer, uses the UIO mechanism to make the network card driver module run in user mode, bypasses the kernel network protocol stack, uses polling and zero copy technology to collect packets from the network card, and uses large pages and CPU affinity mechanisms to improve the performance of application modules in processing packets, thereby saving overhead and achieving the requirements for improving data packet processing performance.

At present, the negotiation process and services of most traditional open source projects rely on the Linux kernel and mature hardware configuration, and the performance is greatly restricted by the hardware performance. The size of the device throughput is mainly determined by the efficiency of the hardware and module algorithms of the network equipment, especially the module algorithm. The inefficiency of the algorithm will greatly reduce the communication volume. The traditional IKE protocol requires multiple (up to 9 round trips) information interactions, and each interaction will perform a large number of data encryption and decryption operations, and the performance of all complex algorithms such as key management depends on and occupies the CPU. Therefore, the number of interactions and the performance of the traditional IKE negotiation algorithm will have a certain impact on the overall performance of the communication. Especially in the case of large-scale concurrent connections and high loads, it may lead to a decrease in communication speed and an increase in latency.

As can be seen from the above, the defects of the prior art are: the current key negotiation method does not support the DPDK architecture, is overly dependent on CPU performance, and its performance is greatly restricted by hardware performance.

Summary of the invention

In view of the defects of the prior art, the present invention provides a key negotiation system and method based on the DPDK architecture, which solves the disadvantage of excessive dependence on CPU performance caused by the open source framework of traditional key negotiation due to the inability to be used based on the DPDK architecture, ensures the synchronization and stable interaction between the modules inside the device, and makes the VPN device more reliable.

In order to solve the technical problem, the technical solution adopted by the present invention is: a key negotiation system based on DPDK architecture, including a management configuration module, a business processing module, a key negotiation module, a DPDK component and a DPDK driver. The management configuration module is respectively connected to the business processing module and the key negotiation module, and is used to send management configuration data to the business management module and the key negotiation module; the business processing module is used for network data transmission and reception, business data encryption and decryption, data encapsulation and decapsulation, and negotiation traffic triggering. The key negotiation module is used to implement key management algorithm, SAD maintenance, SPD maintenance, and IKE configuration. The business processing module is connected to the DPDK component and the DPDK driver in sequence to realize the interaction of network data packets.

Furthermore, the management configuration data includes system configuration and security policies.

The present invention also discloses a key negotiation method based on the DPDK architecture. The method is implemented based on the above key negotiation system and includes the following steps:

S01), key negotiation initiation, the key negotiation module supports two key negotiation initiation modes: automatic trigger and traffic trigger. Traffic trigger is a mandatory option, and automatic trigger is an optional option. If automatic triggering is configured, when the policy is loaded successfully and the entire device starts working, the entire policy table is negotiated in sequence until all policy negotiations and security alliances are issued. If automatic triggering is not configured, the business processing module sends a negotiation initiation request message to the key negotiation module to trigger policy negotiation;

S02), negotiate the interaction of network data packets. The key negotiation module converts the negotiation packet of the initiator into a private format and sends it to the business processing module. After receiving the packet, the business processing module parses the packet, restores it to a standard TCP/IP network packet and sends it to the responder. After the responder completes the processing, it sends the network packet to the business processing module. The business processing module strips off all the header information of the received network packet according to the TCP/IP network message format, retains only the original address, destination address, source port, destination port, and IKE payload information, and hands it over to the key negotiation module for processing in a private format; after multiple interactions, the final SA and SA-SP association relationship are negotiated and sent to the business processing module, which encrypts and decrypts the business data; SA stands for security alliance and SP stands for security policy;

S03), algorithm operation, the encryption algorithm used in the key negotiation process is called by the key negotiation module through the application layer interface, and the operation result is sent to the key negotiation module;

S04), SAD, SPD maintenance, SAD stands for security association database, SPD stands for security policy database, the key negotiation module is responsible for maintaining the SAD table and operating the SAD table, the initial SPD table is issued by the management configuration module and subsequently maintained by the key negotiation module;

S05), key re-negotiation, after key re-negotiation is completed, notify the service processing module to delete the old SA and load the new SA.

The encryption algorithm used in the key negotiation process is integrated in the hardware encryption card, FPGA or soft algorithm, and the key negotiation module calls the encryption algorithm through the application layer interface.

Furthermore, the process of the key negotiation module calling the encryption algorithm in the FPGA to perform algorithm calculation is as follows:

S11), the key negotiation module uses a custom data format to construct a data request packet according to the required computing service type;

S12), the key negotiation module transmits the formed data request packet to the FPGA algorithm idle queue through the application layer interface;

S13), FPGA detects that there is a data request packet in the idle queue, parses the data request packet, performs cryptographic operation according to the request service type identifier, and returns the result;

S14), the key negotiation module receives and parses the response data packet to obtain the result of the request data.

Furthermore, a single SA attribute includes a policy attribute, a sequence number, a life cycle and an SPI. The SPI is a security parameter index in an IPsec message and is used to uniquely mark an SA. The SA sequence number and the SPI of the SA in the same direction are unique and are generated by the key negotiation module and sent to the service processing module. The SAD is stored in a linked list structure and the memory space occupied is dynamically allocated.

Furthermore, operations on SAD, including adding, deleting, and checking, are completed by the key negotiation module;

The specific process of adding a new SA is as follows:

S21), assuming that the policy number is 1, generate an SA number, and the generation rule is: the SA number generated this time is equal to the SA number generated last time plus 1;

S22), generate SPI and obtain the SPI generated by the other party, each device is responsible for generating its inbound SPI, the inbound SPI of the local end is the same as the outbound SPI of the other end, and the outbound SPI of the local end is the same as the inbound SPI of the other end;

S23), calculate the key value, calculate the outbound SA and inbound SA keys, and part of the plain text of the key value comes from the SPI;

S24), search for the outbound SA with SPD number 1, if there is one, delete it, and notify the service processing module to delete it after the set time, and release its number at the same time;

S25), configure outbound SA attributes, part of which comes from policy 1, and the rest is generated by the key negotiation module;

S26), send the outbound SA to the business processing module, and insert the outbound SA into the linked list;

S27) Find the inbound SA with SPD number 1. If there is one, delete it and notify the service processing module to delete it after a set time, and release its number at the same time;

S28), configure new inbound SA attributes, part of which comes from strategy 1, and the rest is generated by the key negotiation module;

S29) Send the inbound SA to the business processing module and insert the inbound SA into the linked list.

Furthermore, in steps S21) and S22), a bitmap method is used to ensure the uniqueness of the SA sequence number and the SPI.

Furthermore, the triggering methods of key renegotiation include packet number triggering and time triggering. The packet number triggering is implemented by the service processing module. When the packet number life cycle is reached, the service processing module sends a renegotiation command to the key negotiation module through the message queue, and the key negotiation module performs renegotiation according to the SPD sequence number therein;

Furthermore, the time trigger is implemented by the key negotiation module. When the SA reaches its life cycle, the key negotiation module automatically initiates key re-negotiation.

Furthermore, in the time-triggered mode, the time for initiating key renegotiation is set to 80% of its life cycle. If a new SA is not negotiated after 100% of the life cycle is reached, the SA is forcibly deleted.

Beneficial effects of the present invention: The present invention solves the problem of over-reliance on CPU performance caused by the inability to use the traditional open source framework of key negotiation based on the DPDK architecture, ensures the synchronization and stable interaction between the modules inside the device, and makes the VPN device more reliable.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a block diagram of a key agreement system based on the DPDK architecture;

FIG2 is a schematic diagram of a key agreement algorithm service call model;

FIG3 is a schematic diagram of a custom protocol for a key negotiation data packet.

Detailed ways

The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

Example 1

This embodiment discloses a key negotiation system based on the DPDK architecture, as shown in FIG1 , including a management configuration module, a business processing module, a key negotiation module, a DPDK component, and a DPDK driver. This system redesigns the key management algorithm, the SAD (Security Association Database), the SPD (Security Policy Database) structure and storage design, the IKE configuration and other functions. The above functions are implemented by the key negotiation module, and the functions of network data transmission and reception, business data decryption and decryption, data encapsulation/decapsulation, negotiation traffic triggering, etc. are handed over to the business processing module for processing. The management configuration module sends the system configuration, security policy and other contents to the business processing module and the key negotiation module. The business processing module is connected to the DPDK component and the DPDK driver in sequence to realize the interaction of network data packets.

Example 2

This embodiment discloses a key negotiation method based on the DPDK architecture. This method is based on the key negotiation system described in Example 1. In order to ensure the overall security and performance of the key negotiation based on the DPDK architecture, the symmetric, asymmetric, and hash algorithms involved are provided with cryptographic services through hardware encryption cards/FPGAs/soft algorithm libraries, thereby reducing the time for single negotiation algorithm operations. The use of the DPDK suite increases the speed of network data packet forwarding and increases the speed of a single negotiation. The present invention is described below from the aspects of key negotiation initiation, negotiation of network data packet interaction, algorithm operation, SAD maintenance, and key renegotiation.

Key negotiation is initiated. The key negotiation module supports two modes: automatic triggering and traffic triggering. Among them, traffic triggering is a mandatory option, and automatic triggering is an optional option. If automatic triggering is configured, when the policy is loaded successfully and the entire system (i.e., the key negotiation system composed of the management configuration module, the business processing module, the key negotiation module, the DPDK component and the DPDK driver described in Example 1) starts working, the entire policy table is negotiated in sequence until all policy negotiations and SA issuance are completed. If automatic triggering is not configured, the business processing module sends a negotiation initiation request message to the key negotiation module to trigger policy negotiation. Automatic triggering is suitable for scenarios with low security requirements, fast negotiation speed and fewer policies. Business processing module triggering is often used in large network environments. In actual use, it can be flexibly selected according to the number of policies. In this embodiment, the process of traffic triggering is: the business processing module queries whether the traffic has been negotiated. If it has not been negotiated, it sends a "negotiation initiation command" to the key negotiation module to trigger the negotiation. Therefore, the initiation point of traffic triggering is the business processing module, and the processing point is the key negotiation module.

Negotiate the interaction of network data packets. The negotiation packet of the local network cryptographic device (initiator) is converted by the key negotiation module into a private format and sent to the business processing module through the message queue. After receiving the packet, the business processing module parses the packet, restores it to a standard TCP/IP network packet and sends it to the peer network cryptographic device (responder). After the peer device completes the processing, it sends the network packet to the business processing module. The business processing module strips off all the header information of the received network packet according to the TCP/IP network message format, retains only the original address, destination address, source port, destination port, and IKE payload information, and hands it over to the key negotiation module for processing in a private format. After multiple interactions, the final SA and SA-SP association relationship are negotiated and sent to the business processing module, which encrypts and decrypts the business data. SA stands for security alliance and SP stands for security policy.

Algorithm operation. The encryption algorithm used in the negotiation process is called by the key negotiation module through the hardware encryption card/FPGA/soft algorithm application layer interface, and the hardware encryption card/FPGA/soft algorithm sends the operation result to the key negotiation module. The encryption and decryption process of network data packets has nothing to do with key negotiation and will not be described here.

SAD/SPD maintenance. The maintenance of SAD is the security core of the entire network cryptographic device. The key negotiation module is responsible for maintaining the SAD table. The attributes of a single SA include not only the attributes of its policy, such as transmission, tunnel, protocol number, etc., but also its unique attributes such as serial number, life cycle, SPI (the security parameter index in the ipsec message, used to uniquely mark an SA). The SA serial number and the SPI of the SA in the same direction are unique. They are all generated by the key negotiation module and sent to the business processing module. Operations on SAD include adding, deleting, and checking, which are all completed by the key negotiation module. SPD maintenance. The key negotiation module will maintain a simple SPD table to establish the association between SA and SP. The SPD table is issued by the management configuration module.

In this embodiment, the storage of SAD adopts a linked list structure, and the occupied memory space adopts a dynamic allocation method. The above method enables SA to perform addition, deletion and query operations more conveniently.

Taking the initiator as an example, the specific process of adding a new SA is as follows:

1) Assuming the policy number is 1, a two-byte SA sequence number (sad_seq) is generated. The SA sequence number (sad_seq) generation rule is: the SA sequence number generated this time is equal to the SA sequence number generated last time plus 1. The bitmap method is used to ensure the uniqueness of the SA sequence number.

In this embodiment, the SA sequence number starts from 1. Each time an SA is generated, its sequence number value increases by 1 until the maximum value we set is reached, and then the SA starts from 1 again. During this period, some SAs must be deleted and their SA sequence numbers are recycled. If 1 is occupied, 2 is used. If the number of SAs reaches the maximum number, it means that all sequence numbers are used, and this negotiation is marked as failed.

2) Generate SPI and obtain the SPI generated by the peer device. Each device generates its inbound SPI and ensures that it is unique. The inbound SPI of the local device is the same as the outbound SPI of the peer device, and the outbound SPI of the local device is the same as the inbound SPI of the peer device. The bitmap method can be used to ensure the uniqueness of the SPI.

3) Calculate the key value. It is necessary to calculate the outbound SA and inbound SA keys, and part of the plaintext of the key value comes from the SPI.

4) Search for the outbound SA with SPD number 1. If there is one, delete it and notify the service processing module to delete it after 3 seconds. The reason for delaying the notification of the service processing module is that the old and new SAs can exist at the same time in the same direction and with the same policy to avoid packet loss. When notifying the service processing module to delete it, its number is released so that it can be occupied later.

5) Configure outbound SA attributes. Some of the outbound SA attributes come from policy 1, and the rest are generated by the key negotiation module.

6) Send the outbound SA to the business processing module and insert the outbound SA into the linked list.

7) Search for the inbound SA with SPD number 1. If there is one, delete it and notify the service processing module to delete it after 3 seconds. The reason for delaying the notification of the service processing module is that the old and new SAs can exist at the same time in the same direction and with the same policy to avoid packet loss. When notifying the service processing module to delete it, its sequence number is released so that it can be occupied later.

8) Configure new inbound SA attributes. Some of the new inbound SA attributes come from policy 1, and the rest are generated by the key negotiation module.

9) Send the inbound SA to the business processing module and insert the inbound SA into the linked list.

Key renegotiation. There are two types of triggers for key renegotiation: packet number trigger and time trigger.

The packet number trigger is managed by the business processing module, because the number of encrypted and decrypted packets is counted by the business processing module. When the packet number life cycle is reached, the business processing module sends a renegotiation command to the key negotiation module through the message queue, and the key negotiation module performs renegotiation based on the SPD sequence number.

The time trigger is controlled by the key negotiation module. When the SA reaches its life cycle, the key negotiation module automatically initiates renegotiation. The time to initiate renegotiation is generally set to 80% of its life cycle. If no new SA is negotiated after 100% of the life cycle is reached, the SA is forcibly deleted.

The re-negotiation process is basically the same as the initial negotiation process. The difference is that after the re-negotiation is completed, the service processing module will be notified to delete the old SA and load the new SA.

Example 3

As shown in Figure 2, the hardware encryption card/FPGA/soft algorithm provides symmetric algorithm services, asymmetric algorithm services, hash function operation services, random number services and other services for the key negotiation module through the application layer interface.

Taking the key negotiation using FPGA algorithm service as an example, the process of the key negotiation module calling the encryption algorithm through the application layer interface is as follows:

S11), the key negotiation module uses a custom data format to construct a data request packet according to the required corresponding operation service type.

S12), the key negotiation module transmits the assembled request packet data to the FPGA algorithm idle queue through the application layer interface.

S13), FPGA detects that there is an algorithm request packet in the idle queue, parses the request packet, performs cryptographic operation according to the request service type identifier, and returns the result.

S14), the key negotiation module receives and parses the response data packet to obtain the result of the request data.

Example 4

In this embodiment, the key negotiation module and the service processing module communicate with each other through the system message queue in a private format.

As shown in Figure 3, the negotiation interaction network data packet adopts a custom private format. For the data sent by the key negotiation module (including the original address, destination address, source port, destination port, and IKE payload information), the business processing module restores it to the standard TCP/IP network message format and sends it out through the network. In the opposite direction, the business processing module strips off all the header information of the received negotiation data packet according to the TCP/IP network message format, retaining only the original address, destination address, source port, destination port, and IKE payload information, and passes it to the key negotiation module for processing in a private format. Through this method, the TCP/IP protocol communication is completely cut off, thereby meeting the needs of key negotiation processing security.

Other data packets exchanged between the key negotiation members and the service processing module also use a private format, including SA delivery, IKE request data packets, etc.

The above description is only the basic principle and preferred embodiments of the present invention. Improvements and substitutions made by those skilled in the art based on the present invention belong to the protection scope of the present invention.

Claims (8)

1. A key negotiation method based on DPDK architecture is characterized in that: the method is realized based on a key negotiation system, wherein the key negotiation system comprises a management configuration module, a service processing module, a key negotiation module, a DPDK component and a DPDK driver, wherein the management configuration module is respectively connected with the service processing module and the key negotiation module and is used for sending management configuration data to the service management module and the key negotiation module, and the management configuration data comprises system configuration and a security policy; the service processing module is used for network data receiving and transmitting, service data encrypting and decrypting, data packaging and decapsulating and negotiating flow triggering, the key negotiation module is used for realizing key management algorithm, SAD maintenance, SPD maintenance and IKE configuration, and the service processing module is connected with the DPDK component and the DPDK drive in sequence and used for realizing interaction of network data packets;

The method comprises the following steps:

s01), initiating key agreement, wherein the key agreement module supports two key agreement initiating modes of automatic triggering and flow triggering, the flow triggering is a necessary option, and the automatic triggering is a configuration item; if automatic triggering is configured, after the strategy is loaded successfully and the whole key negotiation system starts working, sequentially initiating negotiation on the whole strategy table until all strategy negotiation and security alliance issuing are completed, and if automatic triggering is not configured, sending a negotiation initiation request message to the key negotiation module by the service processing module to trigger the strategy negotiation;

S02), negotiating interaction of network data packets, converting the data packets of an initiator into a private format by a key negotiation module and sending the private format to a service processing module, analyzing the data packets after the service processing module receives the data packets, restoring the data packets into standard TCP/IP network data packets and sending the standard TCP/IP network data packets to a responder, sending the network data packets to the service processing module after the responder finishes processing, stripping all header information of the received network data packets according to a TCP/IP network message format by the service processing module, and only reserving original addresses, destination addresses, source ports, destination ports and IKE load information and processing the network data packets by the key negotiation module through the private format; after multiple interactions, negotiating a final SA and SA-SP association relationship, and sending the final SA and SA-SP association relationship to a service processing module, wherein the service processing module encrypts and decrypts service data; wherein SA represents a security association and SP represents a security policy;

s03), algorithm operation, wherein an encryption algorithm used in the key negotiation process is called by the key negotiation module through an application layer interface, and an operation result is sent to the key negotiation module;

S04), SAD and SPD maintenance, wherein SAD represents a security alliance database, SPD represents a security policy database, a key negotiation module is responsible for maintaining SAD tables and operating the SAD tables, and an initial SPD table is issued by a management configuration module and is subsequently maintained by the key negotiation module;

S05), key renegotiation, and after the key renegotiation is completed, notifying a service processing module to delete the old SA and load the new SA.

2. The DPDK architecture based key agreement method according to claim 1, wherein: the encryption algorithm used in the key negotiation process is integrated on a hardware encryption card, an FPGA or a soft algorithm, and the key negotiation module calls the encryption algorithm through an application layer interface.

3. The DPDK architecture based key agreement method according to claim 2, wherein: the key negotiation module calls an encryption algorithm in the FPGA to carry out algorithm operation, and the process comprises the following steps:

S11), the key negotiation module builds a data request packet by using a custom data format according to the required operation service type;

s12), the key negotiation module transmits the built data request packet to the inside of an FPGA algorithm idle queue through an application layer interface;

S13), the FPGA monitors that the idle queue has a data request packet, analyzes the data request packet, carries out password operation according to the request service type identifier and returns a result;

S14), the key negotiation module receives and analyzes the response data packet to obtain the result of the request data.

4. The DPDK architecture based key agreement method according to claim 1, wherein: the single SA attribute comprises a strategy attribute, a serial number, a life cycle and an SPI, wherein the SPI is a security parameter index in an ipsec message and is used for uniquely marking one SA, the SA serial number and the SPI of the SA in the same direction are unique, and the SPI is generated by a key negotiation module and issued to a service processing module; the SAD is stored in a linked list structure, and the occupied memory space is dynamically allocated.

5. The DPDK architecture based key agreement method according to claim 4, wherein: the SAD operation comprises adding, deleting and checking, which are completed by a key negotiation module;

the specific flow of the newly added SA is as follows:

S21), assuming that the policy number is 1, generating an SA number, and the generation rule is: the SA serial number generated at this time is equal to the serial number of the SA generated last time plus 1;

S22), generating SPIs and obtaining SPIs generated by opposite terminal equipment, wherein each opposite terminal equipment generates an inbound SPI thereof, the inbound SPI of the local terminal equipment is identical to the outbound SPI of the opposite terminal equipment, and the outbound SPI of the local terminal equipment is identical to the inbound SPI of the opposite terminal equipment;

S23), calculating a key value, and calculating two keys of the outbound SA and the inbound SA, wherein partial plaintext of the key value is from the SPI;

s24), searching an outbound SA with SPD serial number of 1, deleting if the outbound SA is found, informing a service processing module of deleting after the set time, and releasing the serial number;

s25), configuring an outbound SA attribute, wherein part of the outbound SA attribute is from the strategy 1, and the other part of the outbound SA attribute is generated by a key negotiation module;

S26), issuing an outbound SA to a service processing module, and inserting the outbound SA into a linked list;

s27), searching an inbound SA with SPD sequence number of 1, deleting if the inbound SA is available, notifying a service processing module of deleting after a set time, and releasing the sequence number;

s28), configuring new inbound SA attributes, wherein part of the new inbound SA attributes come from the strategy 1, and the other new inbound SA attributes are generated by a key negotiation module;

S29), issuing the inbound SA to the service processing module, and inserting the inbound SA into the linked list.

6. The DPDK architecture based key agreement method according to claim 5, wherein: steps S21) and S22), uniqueness of the SA sequence number and the SPI is ensured by adopting a bitmap method.

7. The DPDK architecture based key agreement method according to claim 1, wherein: the triggering mode of key renegotiation comprises packet number triggering and time triggering, wherein the packet number triggering is implemented by a service processing module, and when the life cycle of the packet number arrives, the service processing module sends a renegotiation command to a key negotiation module through a message queue, and the key negotiation module renegotiates according to the SPD sequence number in the key negotiation module;

The time triggering is implemented by a key agreement module which automatically initiates a key renegotiation when the SA arrives at the lifecycle.

8. The DPDK architecture based key agreement method according to claim 7, wherein: in the time triggered mode, the time for initiating the key renegotiation is set to 80% of its life cycle, and if no new SA has been negotiated after 100% of the life cycle is reached, the SA is forcedly deleted.