CN118964230A

CN118964230A - Data processing method, storage controller, device and storage medium

Info

Publication number: CN118964230A
Application number: CN202411448715.3A
Authority: CN
Inventors: 易恩镁
Original assignee: Aixin Yuanzhi Semiconductor Co ltd
Current assignee: Aixin Yuanzhi Semiconductor Co ltd
Priority date: 2024-10-17
Filing date: 2024-10-17
Publication date: 2024-11-15
Anticipated expiration: 2044-10-17
Also published as: CN118964230B

Abstract

The present application relates to the field of chip technology, and provides a data processing method, storage controller, device and storage medium, the method comprising: obtaining the ith write instruction; i is an integer greater than or equal to 1; identifying the ith write instruction based on configuration parameters, obtaining the ith data to be protected and the ith write address; the configuration parameters indicate the characteristics of multiple storage blocks; the data to be protected corresponds to the storage blocks one by one; the ith data to be protected includes the ith weight data and the ith frame image data; based on the ith write address and the preset offset address, determining the ith target storage address range; the ith target storage address range indicates a part of the first target storage block; the preset offset address is determined based on the configuration parameters; based on the ith target storage address range, writing the ith data to be protected into the first target storage block. The technical solution of the present application can improve the effective bandwidth of DDR, the operating speed and performance of smart terminals.

Description

Data processing method, storage controller, device and storage medium

Technical Field

The present application relates to the field of chip technologies, and in particular, to a data processing method, a storage controller, a device, and a storage medium.

Background

DDR (Double Data Rate Synchronous Dynamic Random Access Memory, double Rate synchronous dynamic random Access memory) is used to store and process sensor data, image data, algorithm data, and the like.

In the intelligent terminal, the weight data and the image data are frequently written into the DDR, the weight data generally refer to data acquired from a sensor, a camera and other devices, and the data are used for tasks such as target detection, tracking and identification after processing and feature extraction. These weight data are frequently written in DDR in the intelligent terminal for subsequent algorithmic processing and decision making. The image data after collection and preprocessing also need to be written into DDR for image processing, deep learning and other algorithms. By writing the weight data and the image data into the DDR, the intelligent terminal can efficiently perform data processing and calculation.

However, in the current read-write operation process of the DDR, a large amount of invalid data is transmitted, so that the effective bandwidth of the DDR is reduced, and the running speed and performance of the intelligent terminal are further reduced.

Disclosure of Invention

The embodiment of the application provides a data processing method, a memory controller, equipment and a memory medium, which can improve the effective bandwidth of DDR, thereby improving the running speed and performance of an intelligent terminal. The technical scheme is as follows:

according to a first aspect of an embodiment of the present application, there is provided a data processing method, including:

Acquiring an ith writing instruction; i is an integer greater than or equal to 1;

Identifying the ith writing instruction based on the configuration parameters to obtain the ith data to be protected and the ith writing address; the configuration parameter indicates a characteristic of a plurality of memory blocks; a plurality of the memory blocks are part of a memory; the data to be protected corresponds to the storage blocks one by one; the ith data to be protected comprises the ith weight data and the ith frame image data; the ith write address is the first address of the first target storage block; the first target storage block is used for storing the ith data to be protected;

Determining an ith target storage address range based on the ith writing address and a preset offset address; an ith one of the target memory address ranges indicates a portion of the first target memory block; the preset offset address is determined based on the configuration parameters;

and writing the ith data to be protected into the first target storage block based on the ith target storage address range.

In one possible implementation, the configuration parameters include the number of memory blocks, the size of the memory blocks, and a preset memory address range of the memory blocks.

In one possible implementation manner, the determining the ith target storage address range based on the ith write address and a preset offset address includes:

taking the product of the preset offset address and i as an i-th target offset address;

Determining an ith target storage address range based on the ith target offset address and the ith write address; the ith target storage address range is greater than or equal to the ith original storage address range and less than or equal to the preset storage address range of the first target storage block; the ith original storage address range indicates the size of the ith data to be protected.

In one possible implementation, the method further includes:

Processing the ith data to be protected based on the configuration parameters to generate ith target protection data; the ith target protection data comprises the ith data to be protected and the ith mark data; the flag data corresponds to the memory blocks one by one.

In one possible implementation, the ith flag data includes an ith first flag and an ith second flag;

The ith first mark is arranged at the head of the image data of the ith frame; the ith second mark is set at the tail of the image data of the ith frame.

In one possible implementation, the method further includes:

Acquiring a writing process parameter; the write process parameter indicates write normal or write abnormal;

And if the writing process parameter indicates writing abnormality, generating a first interrupt signal.

In one possible implementation, the method further includes:

if the (i+1) th target protection data is not obtained after the error tolerance time is exceeded, generating a second interrupt signal; the error tolerance time is greater than the recovery time; the recovery time is the duration between the time point when the ith target protection data is in error and the time point when the (i+1) th target protection data starts to be written; the (i+1) th target protection data is correct data.

In one possible implementation, the method further includes:

acquiring a kth reading instruction; k is an integer greater than or equal to 1;

Identifying a kth reading instruction based on the configuration parameters to obtain a kth target storage address range;

determining a kth target offset address based on the configuration parameters;

Obtaining a kth reading address based on the kth target storage address range and the kth target offset address; the kth said read address is the first address of the kth said target storage address range; the kth read address indicates a second target memory block;

and reading the kth target protection data from the second target storage block based on the kth read address.

In one possible implementation, the method further includes:

analyzing the kth target protection data to obtain the kth data to be read and the kth mark data;

outputting the kth data to be read if the kth flag data is correct; or (b)

And if the kth sign data is incorrect, not outputting the kth data to be read, and generating a read error signal.

In one possible implementation, the method further includes:

Acquiring a reading process parameter; the reading process parameter indicates reading normal or reading abnormal;

And if the reading process parameter indicates reading abnormality, generating a third interrupt signal.

According to a second aspect of an embodiment of the present application, there is provided a memory controller, the apparatus comprising:

The acquisition module is used for acquiring an ith writing instruction; i is an integer greater than or equal to 1;

The write identification module is used for identifying the ith write instruction based on the configuration parameters to obtain the ith data to be protected and the ith write address; the configuration parameter indicates a characteristic of a plurality of memory blocks; a plurality of the memory blocks are part of a memory; the data to be protected corresponds to the storage blocks one by one; the ith data to be protected comprises the ith weight data and the ith frame image data; the ith write address is the first address of the first target storage block; the first target storage block is used for storing the ith data to be protected;

The address module is used for determining an ith target storage address range based on the ith writing address and a preset offset address; an ith one of the target memory address ranges indicates a portion of the first target memory block; the preset offset address is determined based on the configuration parameters;

And the writing module is used for writing the ith data to be protected into the first target storage block based on the ith target storage address range.

According to a third aspect of embodiments of the present application, there is provided a computer device comprising a processor and a memory for storing at least one program, the at least one program being loaded by the processor and performing the data processing method as described above.

According to a fourth aspect of embodiments of the present application, there is provided a computer-readable storage medium having stored therein at least one program loaded and executed by a processor to implement the data processing method as described above.

In the embodiment of the application, the embodiment of the application provides a data processing method, and each frame of image data is sequentially stored in a designated position, so that the problems of collision and coverage of the image data are avoided, and the safety of the image data is further protected. In addition, compared with the related art, the embodiment of the application does not need to embed error checking and correcting values in the image data, thereby relieving the pressure of bandwidth, improving the performance of an advanced driving auxiliary system and further improving the safety of automatic driving.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic illustration of an implementation environment provided in accordance with an embodiment of the present application;

Fig. 2 is a schematic structural view of an advanced driving assistance system chip provided in the related art;

FIG. 3 is a schematic diagram of a memory controller according to the related art;

FIG. 4 is a schematic diagram of an inline error checking and correction security unit provided in the related art;

FIG. 5 is a schematic diagram of a memory controller controlling DDR read and write without using an inline error check and correction security unit, as provided in the related art;

FIG. 6 is a schematic diagram of a memory controller controlling DDR read and write using an inline error check and correction security unit as provided in the related art;

FIG. 7 is a schematic diagram of a related art error checking and correction value occupying a bandwidth proportion;

FIG. 8 is a flow chart of a data processing method according to an embodiment of the present application;

FIG. 9 is a flow chart of a step 803 provided in accordance with an embodiment of the present application;

FIG. 10 is a schematic diagram of a data store provided by the related art;

FIG. 11 is a schematic diagram of a data store according to an embodiment of the present application;

FIG. 12 is a schematic diagram of a data flow between a memory controller and DDR according to an embodiment of the application;

FIG. 13 is a schematic diagram of error tolerance protection provided in accordance with an embodiment of the present application;

fig. 14 is a schematic diagram showing that an error occurs in the i-th frame image data provided according to an embodiment of the present application;

FIG. 15 is a flow chart of another data processing method according to an embodiment of the present application;

FIG. 16 is a flow chart following step 1505 provided in accordance with an embodiment of the present application;

FIG. 17 is a schematic diagram of a memory controller according to an embodiment of the present application;

Fig. 18 is a schematic structural diagram of a terminal according to an embodiment of the present application;

Fig. 19 is a schematic structural diagram of a server according to an embodiment of the present application.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the present application more apparent, the embodiments of the present application will be described in further detail with reference to the accompanying drawings.

Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the application.

The terms "first," "second," and the like in this disclosure are used for distinguishing between similar elements or items having substantially the same function and function, and it should be understood that there is no logical or chronological dependency between the terms "first," "second," and "n," and that there is no limitation on the amount and order of execution. It will be further understood that, although the following description uses the terms first, second, etc. to describe various elements, these elements should not be limited by the terms.

These terms are only used to distinguish one element from another element. For example, a first action can be referred to as a second action, and similarly, a second action can also be referred to as a first action, without departing from the scope of the various examples. The first action and the second action may both be actions, and in some cases may be separate and distinct actions.

At least one means one or more, and for example, at least one action may be an integer number of actions equal to or greater than one, such as one action, two actions, and three actions. The plurality means two or more, and for example, the plurality of actions may be an integer number of actions equal to or greater than two, such as two actions and three actions.

Fig. 1 is a schematic diagram of an implementation environment provided according to an embodiment of the present application, where the implementation environment may include an intelligent terminal 101 and a server 102.

The intelligent terminals 101 are provided with storage controllers; the DDR is arranged outside the intelligent terminal 101, and the memory controller is used for controlling the read-write operation of the DDR. The memory controller may also be understood as a DDR controller.

The intelligent terminal 101 may be an ADAS (ADVANCED DRIVER ASSISTANCE SYSTEMS, advanced driving assistance system) chip provided with a memory controller, an intelligent cabin, a robot, or the like.

The server 102 may be a server, a server cluster formed by a plurality of servers, or a cloud processing center.

The intelligent terminal 101 is connected to the server 102 through a wired or wireless network.

In some embodiments, the wireless network or the wired network uses standard communication techniques and/or protocols. The network is typically the internet, but may be any network including, but not limited to, a local area network (Local Area Network, LAN), metropolitan area network (Metropolitan Area Network, MAN), wide area network (Wide Area Network, WAN), a mobile, wired or wireless network, a private network, or any combination of virtual private networks. In some embodiments, data exchanged over the network is represented using techniques and/or formats including HyperText Mark-up Language (HTML), extensible markup Language (Extensible Markup Language, XML), and the like. All or some of the links may also be encrypted using conventional encryption techniques such as secure sockets layer (Secure Socket Layer, SSL), transport layer security (Transport Layer Security, TLS), virtual private network (Virtual Private Network, VPN), internet protocol security (Internet Protocol Security, IPsec), etc. In other embodiments, custom and/or dedicated data communication techniques may also be used in place of or in addition to the data communication techniques described above.

Fig. 2 is a schematic structural diagram of an advanced driving assistance system chip provided in the related art.

In the related art, an advanced driving assistance system chip is a basic and advanced technology of an automatic driving technology, and provides necessary environment sensing and assistance control support for the automatic driving system. As shown in fig. 2, an advanced driving assistance System Chip is provided with a SoC (System on Chip), and the SoC is provided with a CPU (Central Processing Unit ), an NPU (Neural Processing Unit, neural network processor), a cache, an input/output interface, and the like.

In the advanced driving assistance system chip, a memory controller and a DDR PHY (PHYSICAL LAYER ) are further provided. The memory controller is used for managing the read-write operation of the DDR, and the DDR chip is located outside the advanced driving assistance system chip and used for storing various data. The DDR PHY converts the data sent by the memory controller into signals conforming to the DDR protocol and sends the signals to the DDR.

DDR is used to store and process sensor data, image data, algorithm data, etc., to support various functions and algorithms of advanced driving assistance system chips. DDR includes a plurality of DDR particles, which represent the smallest memory cells in DDR.

In the advanced driving assistance system chip, the weight data and the image data need to be written frequently to the DDR. The weight data generally refers to data obtained from a sensor, a camera and other devices, and is used for tasks such as target detection, tracking, identification and the like after processing and feature extraction. These weight data are frequently written in the DDR in advanced driving assistance system chips for subsequent algorithmic processing and decision making. And image data representing original pixel information of the image or video data. After image data is collected and preprocessed, the image data also needs to be written into DDR (double data rate) so as to be convenient for algorithms such as image processing, deep learning and the like.

Through writing the weight data and the image data into the DDR, the advanced driving assistance system chip can efficiently perform data processing and calculation, and realize a real-time intelligent driving assistance function, thereby improving driving safety and comfort.

Fig. 3 is a schematic diagram of a structure of a memory controller according to the related art.

As shown in fig. 3, a core processing unit (core) is provided in the memory controller, and a INLINE ECC (Inline Error CHECKING AND correction) security unit is provided in the core component. In addition, the memory controller is also provided with an LBIST (Logic Built-In Self-Test) mechanism, an AXI (Advanced eXtensible Interface ), an APB (ADVANCED PERIPHERAL Bus, advanced peripheral Bus), a Debug mechanism, an arbiter and the like.

Wherein the weight data and the image data are data related to functional safety. The related art is to meet the requirements of the functional safety standard ISO 2626 (International Organization for Standardization 2626) standard of weight data and image data, integrate an inline error checking and correcting safety unit in a memory controller for detecting hardware failure of DDR particles storing weight data and image data. The advanced driver assistance system chip automatically enters a safe state when a DDR particulate hardware failure occurs. For example: the advanced driving assistance system chip is restarted. Then, in this case, the driver is required to control the automated driving system.

Fig. 4 is a schematic diagram of an inline error checking and correcting security unit provided in the related art.

As shown in fig. 4, the in-line error checking and correcting security unit includes an error checking and correcting encoder, an error checking and correcting insertion unit, and an address calculator. The error checking and correcting encoder is used for generating error checking and correcting values; the error checking and correcting insertion unit is used for inserting the error checking and correcting value into the corresponding data.

The in-line error checking and correcting security unit can detect and correct bit errors that may occur during data transmission by embedding error checking and correcting values in the data transmission path. When the weight data and the image data are read from or written to the DDR, the weight data and the image data are checked by the inline error checking and correcting security unit. If an error is found, the memory controller will report the error to the SoC.

However, due to consistency problems of weight data, image data, and error correction code reading and writing, delay of DDR increases, and in particular, weight data and image data are frequently written into DDR.

Fig. 5 is a schematic diagram of a related art memory controller controlling DDR read and write without using an inline error checking and correcting security unit.

As shown in fig. 5, the SoC acquires weight data and image data without using an inline error checking and correcting security unit, and the SoC controls the CPU configuration register; the memory controller generates a write-in instruction according to the configuration of the register, and sends the write-in instruction, the weight data and the image data to the DDR; once the DDR completes the data write, the DDR will signal "write data complete (WRITE DATA READY)". Upon receiving this signal, the memory controller will send the next weight data as well as the image data.

Fig. 6 is a schematic diagram of a memory controller controlling DDR read and write using an inline error checking and correcting security unit, as provided by the related art.

As shown in fig. 6, when the in-line error checking and correcting security unit is used, the SoC acquires weight data and image data, the SoC controls the CPU to configure the register, the memory controller generates a write instruction according to the configuration of the register, transmits the write instruction, the weight data and the image data to the DDR, and simultaneously calculates an error checking and correcting value and transmits the write error checking and correcting instruction and the error checking and correcting value. Only after the DDR finishes writing the weight data, the image data, and the error check and correction value, the "write data complete" and "write error check and correction value complete" signals are issued. The memory controller will not send the next write command, weight data, and image data until it receives both signals. The longer waiting time for the "write data complete" and "write error check and correction value complete" signals causes a larger delay.

Thus, although the inline error checking and correcting security unit ensures the security of the advanced driving assistance system chip, especially in the end-to-end automatic driving system, a higher bandwidth occupation ratio may result when frequent writing of weight data and image data to DDR, the inline error checking and correcting security unit embeds the error checking and correcting value into the image data.

Fig. 7 is a schematic diagram of a bandwidth occupation ratio of error checking and correction values provided by the related art.

As shown in fig. 7, the ratio of the corresponding weight data and image data to the error checking and correcting value occupies a bandwidth of 7:1. that is, frequent writing of DDR weight data and image data results in a large increase in the bandwidth pressure of DDR, with an effective bandwidth reduction of more than 30%. This means that the functional safety of the advanced driving assistance system chip cannot fully enable the inline error checking and correcting safety unit to protect data, because the effective bandwidth of the DDR is reduced, packet loss may be caused, and even, a part of functions of the advanced driving assistance system chip cannot be operated normally, thereby reducing the performance of the advanced driving assistance system chip, affecting the automatic driving function and safety, and ultimately constituting a potential threat to the safety of the driver.

In order to solve the problems in the related art, an embodiment of the present application provides a data processing method.

Fig. 8 is a flowchart of a data processing method according to an embodiment of the present application, and as shown in fig. 8, in the embodiment of the present application, a process of writing data to a DDR is described by taking an application to an advanced driving assistance system chip having a memory controller as an example. The method comprises the following steps:

in step 801, the advanced driving assistance system chip acquires an i-th write instruction; i is an integer greater than or equal to 1.

In some embodiments, a portion of the memory region from the DDR is required to be selected prior to step 801, the portion of the memory region being used to store data that needs to be protected. The partial memory area is divided into a plurality of memory blocks of the same size.

In one example, for image data to be protected, it may be determined according to actual needs, that is, any one of the memory blocks may be designated as the first memory block. The size of the storage area and the size of each storage block can be determined according to actual needs.

In one example, the partial memory region may include memory addresses that are either contiguous or non-contiguous. I.e. the head address and the tail address of the different memory blocks may be consecutive or not.

For example: one DDR has a size of 1G, 300M is selected as a storage area, and the storage area of 300M is divided into 6 storage blocks with the same size. I.e. each memory block has a size of 50M.

In some embodiments, the configuration parameters include the number of memory blocks, the size of the memory blocks, and a preset memory address range for the memory blocks.

For example: the preset memory address range of the memory block is 0-50M.

It should be noted that, the plurality of memory blocks themselves have no sequence, and the use sequence of each memory block may be configured according to actual needs. Namely, matching corresponding first target storage blocks for each frame of image data according to actual needs.

In some embodiments, the CPU configures the registers based on the configuration parameters when acquiring a set of weight data as well as image data. Optionally, if the weight data and the image data are data to be protected, the CPU configures a write address for the weight data and the image data; the write address indicates any one of the unoccupied memory blocks. That is, the CPU writes the set of weight data, image data, and information such as the write address into the register.

In some embodiments, the advanced driving assistance system chip retrieves the write instruction from the register.

In step 802, the advanced driving assistance system chip identifies the ith write instruction based on the configuration parameter, and obtains the ith data to be protected and the ith write address; the configuration parameter indicates a characteristic of the plurality of memory blocks; the plurality of memory blocks are part of a memory; the data to be protected corresponds to the storage blocks one by one; the ith data to be protected comprises the ith weight data and the ith frame image data; the ith write address is the first address of the first target memory block; the first target storage block is used for storing the ith data to be protected.

It is to be understood that each memory block is used for storing one frame of image data to be protected in order to avoid overwriting or collision with the i-th frame of image data when writing the i+1th frame of image data into the corresponding target memory block.

In some embodiments, the advanced driving assistance system chip identifies a write address in the write instruction, the write address and the memory block being in one-to-one correspondence. That is, after the writing address is identified, if the writing address indicates any one memory block, the data corresponding to the writing instruction is the data to be protected. Or if the write address does not fall into any storage block of the storage area, the data corresponding to the write instruction is not the data to be protected.

In one example, if the data corresponding to the write instruction is not the data to be protected, the data corresponding to the write instruction and the write address are directly sent to the DDR without executing step 803 and step 804.

In some embodiments, the write instruction includes data information to be protected, write address information, and a data source. Wherein, the data sources comprise NPU or CPU, etc.

For example: when the data source is an NPU, the data identified based on the write instruction is data that does not require protection. Or the data source is a CPU, the data identified based on the write instruction is the data that needs protection.

That is, in the embodiment of the present application, by identifying the data itself, the write address, and the source of the data, it can be determined whether the data corresponding to the write instruction is the data that needs protection.

In some embodiments, the advanced driving assistance system chip sequentially stores the image data in order. Each memory block is used to store one frame of image data. That is, the image data of the i-th frame image is stored first, and then the image data of the i+1-th frame image is stored. Thereby avoiding the problem of disorder in the process of storing images.

In step 803, the advanced driving assistance system chip determines an i-th target storage address range based on the i-th write address and a preset offset address; the ith target memory address range indicates a portion of the first target memory block; the preset offset address is determined based on the configuration parameters.

Fig. 9 is a schematic flow chart of step 803 provided according to an embodiment of the present application.

In some embodiments, the step 803 may include the following steps 8031 and 8032:

in step 8031, the advanced driving assistance system chip uses the product of the preset offset address and i as the i-th target offset address.

In some embodiments, in storing the image data, in order to avoid the i+1st frame image data from overlapping the t frame image, the target offset addresses corresponding to the t+1st frame image data and the t frame image data are not identical. That is, the first target memory block storing the i-th frame image data is not identical to the target memory block storing the i+1th frame image.

In one example, the first memory block is used to store the ith frame of image data. The second storage block is used for storing the (i+1) th frame of image data. The third storage block is used to store the i+2th frame image data.

In some embodiments, it may be necessary to write the i-th frame image data to the same first target memory block in several times, depending on the size of the bandwidth. Wherein, the sub-target offset addresses corresponding to each part of data in the ith frame of image data are the same. The sub-write addresses corresponding to each part of data in the ith frame of image data are different, so that collision or coverage of the data is avoided.

In one example, the ith write instruction may include a plurality of sub-write instructions. A part of the i-th frame image data will be explained as an example. The sub-target storage address range of the partial data can be obtained by a sub-write address and a preset offset address. The sub-write address is the first address of the part of data in the first target storage block, and is obtained through a sub-write instruction.

In one example, the sub-target storage address range for the portion of data may be found by the following formula:

sub-target memory address range = sub-write address + i x preset offset address.

For example, each memory block has a size of 50M, and the i-th frame image data has a size of 5M, the i-th weight data has a size of 10M, and the preset offset address is 30M.

For another example, the i-th frame image data is written in the first target storage block in a plurality of times.

First partial data is written for the first time, second partial data is written for the second time, third partial data is written for the third time, and so on.

Specifically, the sub-target storage address range corresponding to the first partial data=the sub-write address +i×30m corresponding to the first partial data.

Sub-target storage address range corresponding to the second partial data=sub-write address+i×30m corresponding to the second partial data.

A sub-target storage address range corresponding to the third partial data=a sub-write address+i×30m corresponding to the third partial data, and the like.

That is, the sub-write address corresponding to the first portion of data, the sub-write address corresponding to the second portion of data, and the sub-write address corresponding to the third portion of data are different.

Similarly, a sub-target storage address range corresponding to each part of data in the ith frame of image data can be obtained.

Also for example: 128 bits of target protection data are written to the DDR at a time.

In one example, each part of data in the i-th frame of image data is written in turn to the DDR in order. For example, in a left to right, top to bottom order, etc.

In step 8032, the advanced driving assistance system chip determines an i-th target storage address range based on the i-th target offset address and the i-th write address; the ith target storage address range is greater than or equal to the ith original storage address range and less than or equal to the preset storage address range of the first target storage block; the i-th original storage address range indicates the size of the i-th data to be protected.

That is, in the embodiment of the present application, after the first target storage block stores the data to be protected, there is still a remaining storage space, so as to form a buffer space, so as to avoid packet loss or store related feature data into the first target storage block.

Fig. 10 is a schematic diagram of a data storage structure provided in the related art.

As shown in fig. 10, after the virtual address of the i-th frame image data is converted into a physical address, it is stored in the physical address. Wherein the physical address is any free memory space in the DDR. Such a method of not specifying a storage address of data is prone to data collision or data coverage, and the security of the stored data is reduced.

Fig. 11 is a schematic diagram of a data storage structure according to an embodiment of the present application.

The structure of the data store of the present application is described below with reference to fig. 11.

In some embodiments, the write address and the target offset address are designated for the data to be protected, and the data to be protected is stored in the designated position, so that the problem that the data to be protected collides with or covers the data stored before is avoided, and the safety of the data is improved, namely the data to be protected is protected.

In addition, according to the embodiment of the application, the data to be protected are arranged in sequence according to the designated addresses, namely, the image data are stored frame by frame, so that the problem that the middle image data are disordered due to the fact that a large amount of image data are stored is avoided, namely, the image data are more difficult to damage or destroy.

In some embodiments, following step 803, head-to-tail protection is also performed on the ith data to be protected; the head-to-tail protection of the ith data to be protected can be performed in the following implementation manner.

Processing the ith data to be protected based on the configuration parameters to generate the ith target protection data; the ith target protection data comprises the ith data to be protected and the ith mark data; the flag data corresponds to the memory blocks one by one.

In one example, the ith flag data includes an ith first flag and an ith second flag; the ith first flag is set in the header of the ith frame image data; the ith second flag is set at the end of the ith frame image data.

For example: the first flag of the first data to be protected is "001", and the second flag of the first data to be protected is "001".

According to the embodiment of the application, the integrity and the safety of the ith data to be protected are enhanced by protecting the ith data to be protected head and tail.

In step 804, the advanced driving assistance system chip writes the ith data to be protected to the first target memory block based on the ith target memory address range.

In some embodiments, since the ith target offset address and the ith write address are both virtual addresses, before writing the ith data to be protected to a specified location in the DDR, the ith target offset address and the ith write address need to be converted into the ith physical address, and then the ith data to be protected is stored to a storage location in the DDR corresponding to the ith physical address, i.e. address mapping is performed again for the ith data to be protected, so that the ith data to be protected is written to the first target storage block.

In some embodiments, the memory controller sends the ith data to be protected and the ith physical address to the DDR through the interface, and the DDR writes the ith data to be protected into the first target memory block corresponding to the ith physical address.

For example, the interface is implemented based on DDR DFI (DDR PHY INTERFACE, DDR physical layer interface).

Fig. 12 is a schematic diagram of a data flow between a memory controller and a DDR according to an embodiment of the present application.

The data flow between the memory controller and the DDR is illustrated in connection with FIG. 12.

In some embodiments, once the memory controller sends the ith physical address and the ith data to be protected to the DDR, the DDR completes writing the ith data to be protected, and the DDR will send a "write data complete" signal; the memory controller will send the i+1th physical address and the i+1th protected data. The writing of error checking and correcting values and the consistency of waiting data and writing of error checking and correcting values is reduced compared to an inline error checking and correcting security unit.

Thus, embodiments of the present application do not substantially increase bandwidth pressure in the data flow between the memory controller and the DDR, thereby significantly relieving bandwidth pressure compared to an inline error checking and correction security unit.

In some embodiments, in performing steps 801 through 802, the write process is monitored by the following implementation:

acquiring a writing process parameter; the write process parameter indicates write normal or write abnormal; if the write process parameter indicates a write exception, a first interrupt signal is generated.

For example: writing normal includes no timeout, no illegal jump, normal state, etc.

The write exception includes at least one of a timeout, an illegal jump, and a state exception.

Uploading a first interrupt signal to the SoC; the SoC may stop the operation of the advanced driving assistance system chip based on the first interrupt signal.

FIG. 13 is a schematic diagram of error tolerance protection according to an embodiment of the present application.

Error tolerance protection is illustrated in connection with fig. 13.

In some embodiments, error tolerance protection may be implemented by:

If the (i+1) th target protection data is not obtained after the error tolerance time is exceeded, generating a second interrupt signal; error tolerance time is greater than recovery time; the recovery time is the time length from the time point when the ith target protection data is in error to the time point when the (i+1) th target protection data starts to be written; the (i+1) th target protection data is correct data.

Therefore, since the recovery time is less than the error tolerance time, the second interrupt signal is not triggered; the advanced driving assistance system chip can process data according to the correct (i+1) th target protection data in error tolerance time so as to ensure the safety of an automatic driving function. The data processing may be understood as that the advanced driving assistance system chip may acquire the feature value according to the correct i+1th target protection data within the error tolerance time, and so on.

In one example, the memory controller diagnoses the correctness of each frame of image data at all times during the execution of the error tolerance protection; each diagnosis lasts for a period of time, i.e. a diagnosis time.

Diagnosing the correctness of each frame of image data can be understood as diagnosing whether each frame of image data has bit flipping, if so, the ith frame of image data. Generating a write-in error signal when the ith frame of image data is in error, reporting the write-in error signal to the SoC, and judging a data result by the advanced driving assistance system chip according to the following frame of image data of the ith frame of image data; that is, the advanced driving assistance system chip ignores the erroneous frame image data when judging the data result.

Optionally, the SoC accumulates the continuous number of the received write error signals, and if the continuous number of the write error signals reaches the preset number, the SoC may control the advanced driving assistance system chip to stop running.

For example: the preset number is 3, etc.

Fig. 14 is a schematic diagram showing that an error occurs in the i-th frame image data provided according to an embodiment of the present application.

Next, an exemplary description will be given of an error occurrence of the i-th frame image data with reference to fig. 14.

In some embodiments, after the error tolerance time is greater than the recovery time, the i+1st frame image data in the image data stream starts to be written after the error occurs in the i+1st frame image data, and the i+1st frame image data is normal, i.e. the i+2st frame image data is normal, i.e. the problem of single point failure does not occur, so that the advanced driving assistance system chip does not stop running because the time is less than the error tolerance time.

It should be understood that a single point failure refers to the existence of a critical point or component in the system that, if it fails or fails, will cause the entire system or a major portion of the system to fail or cease to function.

Optionally, when the SoC receives three consecutive frames of image data and errors occur, the advanced driving assistance system chip may stop running.

Optionally, when the SoC receives the second interrupt signal, the advanced driving assistance system chip may stop running.

In some embodiments, execution of the data processing method may be turned on or off by configuration. The execution of the data processing method does not affect the bus function of the chip, and no matter the execution of the data processing method is opened or closed, the mapping of the bus is consistent with the bandwidth, so that the chip of the advanced driving assistance system is determined not to be negatively affected in the process of executing the data processing method, and meanwhile, the safety of data is ensured.

It will be appreciated that the embedded ECC value in the image data by the inline error checking and correcting security unit causes the image data to be discontinuous, i.e. a portion of the addresses correspond to the image data and a portion of the addresses correspond to the error checking and correcting value, thereby causing the physical addresses to be scrambled and the mapping of the bus to be inconsistent with the bandwidth. In the embodiment of the application, the image data is continuously stored, and the mapping of the bus is consistent with the bandwidth.

According to the embodiment of the application, the image data of each frame is sequentially stored in the designated position, so that the problems of collision and coverage of the image data are avoided, and the safety of the image data is further protected. In addition, compared with the related art, the embodiment of the application does not need to embed error checking and correcting values in the image data, thereby relieving the pressure of bandwidth, improving the performance of an advanced driving auxiliary system and further improving the safety of automatic driving.

Fig. 15 is a flowchart of another data processing method according to an embodiment of the present application, as shown in fig. 15, in which a process of reading data from a DDR is described as an example of application to an advanced driving assistance system chip having a memory controller. The method comprises the following steps:

in step 1501, the advanced driving assistance system chip acquires a kth read instruction; k is an integer greater than or equal to 1.

In some embodiments, after the advanced driving assistance system chip stores the preset number of frames of image data, the preset number of frames of image data is read from the DDR to perform subsequent calculation, so as to provide a driving strategy for the autopilot system, and further ensure the safety of the driver.

In some embodiments, the advanced driving assistance system chip reads the image data in sequence when reading the image data, thereby preventing an out-of-order problem in the process of reading the image data.

For example: the kth read instruction corresponds to the kth frame of image data.

In step 1502, the advanced driving assistance system chip identifies a kth read instruction based on the configuration parameter, resulting in a kth target storage address range.

In some embodiments, the memory controller may determine whether the kth read instruction indicates image data that needs to be protected based on the configuration parameters. Alternatively, if the kth read instruction indicates that the image data is not to be protected, the kth virtual address is directly converted into the kth physical address and then sent to the DDR. Optionally, if the kth read instruction indicates that the image data needs to be protected, the kth target storage address range is obtained based on the kth virtual address.

It should be explained that the memory controller judges whether the kth image data is the image data to be protected or not by the kth virtual address.

In step 1503, the advanced driving assistance system chip determines the kth target offset address based on the configuration parameters.

In some embodiments, the memory controller calculates a kth offset address from k and a preset offset address.

For example: kth offset address=k×preset offset address.

In step 1504, the advanced driving assistance system chip obtains a kth read address based on the kth target storage address range and the kth target offset address; the kth read address is the first address of the kth target memory address range; the kth read address indicates the second target memory block.

In some embodiments, the memory controller treats the difference between the kth target memory address range and the kth target offset address as the kth read address.

For example: kth read address = kth target memory address range-kth target offset address.

That is, the kth read address=kth target memory address range-k×preset offset address.

In step 1505, the advanced driving assistance system chip reads the kth target protection data from the second target storage block based on the kth read address.

In some embodiments, since the kth read address is a virtual address, the memory controller translates the kth virtual address to the kth physical address, and sends the kth physical address to the DDR.

The DDR determines a second target memory block based on the kth physical address, and then reads all data stored in the second target memory block.

Fig. 16 is a flowchart illustrating a process after step 1505 according to an embodiment of the present application.

The flow after step 1505 is exemplarily described with reference to fig. 16.

In some embodiments, after step 1505 is performed, the following steps 1506 and 1507a or 1507b are also performed:

In step 1506, the advanced driving assistance system chip parses the kth target protection data to obtain the kth data to be read and the kth flag data.

In step 1507a, if the kth flag data is correct, outputting the kth data to be read; or (b)

In step 1507b, if the kth flag data is incorrect, the kth data to be read is not output, and a read error signal is generated.

In some embodiments, the kth flag data includes a kth first flag and a kth second flag. In order to ensure the correctness of the data, before the storage controller outputs the kth target protection data to the SoC, the kth target protection data is subjected to head-to-tail analysis, and the kth first mark is analyzed from the head of the kth target protection data; and the tail of the kth target protection data analyzes the kth second mark.

And if the kth first mark and the kth second mark are matched with the second target storage block, outputting the kth data to be read by the storage controller. Or if any one of the kth first mark and the kth second mark is not matched with the second target storage block, the storage controller directly ignores the kth read data, generates a read error signal and uploads the read error signal to the SoC.

In one example, the SoC accumulates the received error signals, and if the read error signals are continuously received for a preset number of times, the advanced driving assistance system chip is controlled to stop running.

For example: the preset number of times is 3, etc.

In some embodiments, the memory controller, in performing steps 1501 to 1505, monitors the read process by:

Acquiring a reading process parameter; the reading process parameter indicates reading normal or reading abnormal; if the reading process parameter indicates that the reading is abnormal, a third interrupt signal is generated.

For example: reading normal includes no timeout, no illegal jump, normal status, etc.

The read exception includes at least one of a timeout, an illegal jump, and a state exception.

Wherein the memory controller sends a third interrupt signal to the SoC. The SoC controls the advanced driving assistance system chip to stop operating based on the third interrupt signal.

In one example, the image data and corresponding weight data are subsequently calculated based on a pre-set model. For example: the preset module is a neural network module. The neural network model performs feature extraction on the image data based on the weight data.

According to the embodiment of the application, each frame of image data is sequentially stored in the designated position, so that the problems of collision and coverage of the image data are avoided, and the safety of the image data is further protected. In addition, compared with the related art, the embodiment of the application does not need to embed error checking and correcting values in the image data, thereby relieving the pressure of bandwidth, improving the performance of an advanced driving auxiliary system and further improving the safety of automatic driving.

Fig. 17 is a schematic structural diagram of a memory controller 1700 according to an embodiment of the present application, where the memory controller 1700 includes:

An obtaining module 1701, configured to obtain an ith write instruction; i is an integer greater than or equal to 1;

The write identification module 1702 is configured to identify an ith write instruction based on the configuration parameter, to obtain an ith data to be protected and an ith write address; the configuration parameter indicates a characteristic of the plurality of memory blocks; the plurality of memory blocks are part of a memory; the data to be protected corresponds to the storage blocks one by one; the ith data to be protected comprises the ith weight data and the ith frame image data; the ith write address is the first address of the first target memory block; the first target storage block is used for storing the ith data to be protected;

An address module 1703, configured to determine an ith target storage address range based on the ith write address and a preset offset address; the ith target memory address range indicates a portion of the first target memory block; the preset offset address is determined based on the configuration parameters;

The writing module 1704 is configured to write the ith data to be protected into the first target storage block based on the ith target storage address range.

The acquisition module 1701, the write identification module 1702, the address module 1703, and the write module 1704, which are part of the memory controller, are all in hardware.

The memory controller 1700 further includes a configuration module configured to generate configuration parameters.

In some embodiments, the address module 1703 is further configured to:

taking the product of the preset offset address and i as the i-th target offset address;

determining an ith target storage address range based on the ith target offset address and the ith write address; the ith target storage address range is greater than or equal to the ith original storage address range and less than or equal to the preset storage address range of the first target storage block; the i-th original storage address range indicates the size of the i-th data to be protected.

In some embodiments, the memory controller 1700 is further configured to:

The protection module is used for processing the ith data to be protected based on the configuration parameters and generating the ith target protection data; the ith target protection data comprises the ith data to be protected and the ith mark data; the flag data corresponds to the memory blocks one by one.

In some embodiments, the ith flag data includes an ith first flag and an ith second flag;

The ith first flag is set in the header of the ith frame image data; the ith second flag is set at the end of the ith frame image data.

In some embodiments, the memory controller 1700 is further configured to:

if the write process parameter indicates a write exception, a first interrupt signal is generated.

In some embodiments, the memory controller 1700 is further configured to:

Determining a kth target offset address based on the configuration parameters;

Obtaining a kth reading address based on a kth target storage address range and a kth target offset address; the kth read address is the first address of the kth target memory address range; the kth read address indicates a second target memory block;

In some embodiments, the memory controller 1700 is further configured to:

the analysis module is used for analyzing the kth target protection data to obtain the kth data to be read and the kth mark data;

if the kth flag data is correct, outputting the kth data to be read; or (b)

If the kth flag data is incorrect, the kth data to be read is not output, and a read error signal is generated.

In some embodiments, the memory controller 1700 is further configured to:

If the reading process parameter indicates that the reading is abnormal, a third interrupt signal is generated.

It should be noted that: the storage controller provided in the above embodiment only illustrates the division of the above functional modules when executing the corresponding steps, and in practical application, the above functional allocation may be performed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules, so as to complete all or part of the functions described above. In addition, the storage controller and the data processing method provided in the foregoing embodiments belong to the same concept, and specific implementation processes of the storage controller and the data processing method are detailed in the method embodiments and are not repeated herein.

Embodiments of the present application also provide a computer device comprising a processor, a memory and a computer program stored in the memory and configured to be executed by the processor, the processor implementing the method as above when executing the computer program.

Taking a computer device as an example of a terminal, fig. 18 is a schematic structural diagram of a terminal provided in an embodiment of the present application, referring to fig. 18, a terminal 1800 may be: a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III, motion picture expert compression standard audio plane 3), an MP4 (Moving Picture Experts Group Audio Layer IV, motion picture expert compression standard audio plane 4) player, a notebook computer, or a desktop computer. The terminal 1800 may also be referred to as a user device, portable terminal, laptop terminal, desktop terminal, or the like.

In general, the terminal 1800 includes: a processor 1801 and a memory 1802.

Processor 1801 may include one or more processing cores, such as a 4-core processor, a 5-core processor, and the like. The processor 1801 may be implemented in at least one hardware form of DSP (DIGITAL SIGNAL Processing), FPGA (Field-Programmable gate array) GATE ARRAY, PLA (Programmable Logic Array ). The processor 1801 may also include a main processor and a coprocessor, where the main processor is a processor for processing data in an awake state, and is also called a CPU (Central Processing Unit ); a coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 1801 may be integrated with a GPU (Graphics Processing Unit, image processor) for rendering and drawing of content required to be displayed by the display screen. In some embodiments, the processor 1801 may also include an AI (ARTIFICIAL INTELLIGENCE ) processor for processing computing operations related to machine learning.

The memory 1802 may include one or more computer-readable storage media, which may be non-transitory. The memory 1802 may also include high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In some embodiments, a non-transitory computer readable storage medium in memory 1802 is used to store at least one program code for execution by processor 1801 to implement the processes provided in a method embodiment of the present application for terminal execution in the above-described methods.

In some embodiments, the terminal 1800 may also optionally include: a peripheral interface 1803 and at least one peripheral. The processor 1801, memory 1802, and peripheral interface 1803 may be connected by a bus or signal line. The individual peripheral devices may be connected to the peripheral device interface 1803 by buses, signal lines or circuit boards. Specifically, the peripheral device includes: at least one of a display 1804, a camera assembly 1805, audio circuitry 1806, and a power supply 1807.

The peripheral interface 1803 may be used to connect I/O (Input/Output) related at least one peripheral device to the processor 1801 and memory 1802. In some embodiments, processor 1801, memory 1802, and peripheral interface 1803 are integrated on the same chip or circuit board; in some other embodiments, either or both of the processor 1801, memory 1802, and peripheral interface 1803 may be implemented on separate chips or circuit boards, as embodiments of the present application are not limited in this respect.

The display 1804 is used to display a UI (User Interface, user page). The UI may include graphics, text, icons, video, and any combination thereof. When the display 1804 is a touch display, the display 1804 also has the ability to collect touch signals at or above the surface of the display 1804. The touch signal may be input as a control signal to the processor 1801 for processing. At this point, the display 1804 may also be used to provide virtual buttons and/or a virtual keyboard, also referred to as soft buttons and/or a soft keyboard. In some embodiments, the display 1804 may be one and disposed on the front panel of the terminal 1800; in other embodiments, the display 1804 may be at least two, disposed on different surfaces of the terminal 1800 or in a folded configuration; in other embodiments, the display 1804 may be a flexible display disposed on a curved surface or a folded surface of the terminal 1800. Even more, the display 1804 may be arranged in a non-rectangular irregular pattern, i.e., a shaped screen. The display 1804 may be made of LCD (Liquid CRYSTAL DISPLAY), OLED (Organic Light-Emitting Diode) or other materials.

The camera assembly 1805 is used to capture images or video. In some embodiments, the camera assembly 1805 includes a front camera and a rear camera. Typically, the front camera is disposed on the front panel of the terminal and the rear camera is disposed on the rear surface of the terminal. In some embodiments, the at least two rear cameras are any one of a main camera, a depth camera, a wide-angle camera and a tele camera, so as to realize that the main camera and the depth camera are fused to realize a background blurring function, and the main camera and the wide-angle camera are fused to realize a panoramic shooting and Virtual Reality (VR) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 1805 may also include a flash. The flash lamp can be a single-color temperature flash lamp or a double-color temperature flash lamp. The dual-color temperature flash lamp refers to a combination of a warm light flash lamp and a cold light flash lamp, and can be used for light compensation under different color temperatures.

The audio circuitry 1806 may include a microphone and a speaker. The microphone is used for collecting sound waves of users and the environment, and converting the sound waves into electric signals to be input to the processor 1801 for processing. For stereo acquisition or noise reduction purposes, the microphone may be multiple, and disposed at different locations of the terminal 1800. The microphone may also be an array microphone or an omni-directional pickup microphone. The speaker is used to convert electrical signals from the processor 1801 into sound waves. The speaker may be a conventional thin film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, not only the electric signal can be converted into a sound wave audible to humans, but also the electric signal can be converted into a sound wave inaudible to humans for ranging and other purposes. In some embodiments, the audio circuitry 1806 may also include a headphone jack.

A power supply 1807 is used to power the various components in the terminal 1800. The power supply 1807 may be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 1807 includes a rechargeable battery, the rechargeable battery may support wired or wireless charging. The rechargeable battery may also be used to support fast charge technology.

Those skilled in the art will appreciate that the structure shown in fig. 18 is not limiting and may include more or fewer components than shown, or may combine certain components, or may employ a different arrangement of components.

Taking a computer device as a server as an example, fig. 19 is a schematic structural diagram of a server according to an embodiment of the present application, where the server 1900 may have a relatively large difference due to different configurations or performances, and may include one or more processors (Central Processing Units, CPUs) 1901 and one or more memories 1902, where at least one computer program is stored in the one or more memories 1902, and the at least one computer program is loaded and executed by the one or more processors 1901 to implement the above data processing method. Of course, the server 1900 may also have a wired or wireless network interface, a keyboard, an input/output interface, and other components for implementing the functions of the device, which are not described herein.

The embodiment of the application also provides a computer readable storage medium, which comprises a stored computer program, wherein the computer readable storage medium is controlled to execute the method for generating the image processing model by the device when the computer program runs. Alternatively, the computer readable storage medium may be a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a Compact-Disc Read-Only Memory (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, and the like.

It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program for instructing relevant hardware, where the program may be stored in a computer readable storage medium, and the storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.

The foregoing description of the preferred embodiments of the present application is not intended to limit the application, but rather, the application is to be construed as limited to the appended claims.

Claims

1. A method of data processing, comprising:

2. The method of claim 1, wherein the configuration parameters include a number of the memory blocks, a size of the memory blocks, and a preset memory address range of the memory blocks.

3. The method of claim 1, wherein the determining the i-th target storage address range based on the i-th write address and a preset offset address comprises:

4. A method according to claim 3, characterized in that the method further comprises:

5. The method of claim 4, wherein the ith said flag data comprises an ith first flag and an ith second flag;

6. The method according to claim 1, characterized in that the method further comprises:

7. The method according to claim 1, characterized in that the method further comprises:

8. The method according to claim 4, further comprising:

determining a kth target offset address based on the configuration parameters;

9. The method of claim 8, wherein the method further comprises:

outputting the kth data to be read if the kth flag data is correct; or (b)

10. The method of claim 8, wherein the method further comprises:

11. A memory controller, comprising:

12. A computer device, characterized in that it comprises a processor and a memory for storing at least one program, which is loaded by the processor and which performs the data processing method according to any of claims 1 to 10.

13. A computer readable storage medium, characterized in that at least one program is stored in the computer readable storage medium, the at least one program being loaded and executed by a processor to implement the data processing method of any one of claims 1 to 10.