CN117354442B - Method for conveniently adding station logo on LED display screen - Google Patents
Method for conveniently adding station logo on LED display screen Download PDFInfo
- Publication number
- CN117354442B CN117354442B CN202311478324.1A CN202311478324A CN117354442B CN 117354442 B CN117354442 B CN 117354442B CN 202311478324 A CN202311478324 A CN 202311478324A CN 117354442 B CN117354442 B CN 117354442B
- Authority
- CN
- China
- Prior art keywords
- data
- display screen
- led display
- station
- fastcgi
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/44—Receiver circuitry for the reception of television signals according to analogue transmission standards
- H04N5/445—Receiver circuitry for the reception of television signals according to analogue transmission standards for displaying additional information
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/44—Receiver circuitry for the reception of television signals according to analogue transmission standards
- H04N5/445—Receiver circuitry for the reception of television signals according to analogue transmission standards for displaying additional information
- H04N5/44504—Circuit details of the additional information generator, e.g. details of the character or graphics signal generator, overlay mixing circuits
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Graphics (AREA)
- Controls And Circuits For Display Device (AREA)
- Control Of Indicators Other Than Cathode Ray Tubes (AREA)
Abstract
The invention discloses a method for conveniently adding station marks on an LED display screen, which comprises the following steps: receiving an HTTP request and processing the HTTP request by a FastCGI process, wherein the HTTP request comprises a data packet for representing a station logo; analyzing the station caption data packet, and carrying out network interaction with a driving program used for driving the LED display screen on the control panel; the driver of the control panel analyzes the data part of the actual characterization station caption to obtain the analyzed station caption data; loading the characters of the station caption data by adopting a FreeType library, rendering the characters of the station caption data one by one, generating character bitmap data, converting the character bitmap data into pixel data, and rearranging the pixel data based on the pixel position offset; and then, storing the final character bitmap data subjected to the position offset into a file. And finally, periodically reading the bitmap data from the memory, and sending the bitmap data to the LED display screen for display. The invention has higher flexibility, lower resource consumption and higher efficiency.
Description
Technical Field
The invention relates to the technical field of adding station marks in videos, in particular to a method for conveniently adding station marks on an LED display screen.
Background
The LED display screen is widely applied to various occasions such as advertisements, public information release, field performances and the like. For a certain purpose, a station logo is often added on the LED display screen to achieve a certain purpose of propaganda. For example, the LED display screen of a certain market adds the name of the market as a station logo in all or some video advertisements or information release contents to promote the video advertisements or information invention contents to be released for the market. In order to add station logos to video played by LED displays, the prior art is generally implemented in two ways. First, the station marks are added in the process of making the target video to which the station marks need to be added, for example, the station marks are added in the process of making the target video by adopting the video software Adobe Premiere Pro. This implementation has mainly a detailed disadvantage: the station marks need to be added in advance in the production stage of the target video, and once the video is produced, the station marks are difficult to be added again, which means that the target video which is produced and is applied to the LED display screen already begins, and the station marks cannot be added due to temporary requirements or new requirements, so that the application is inconvenient. If the station logo is required to be added after the video is manufactured, the target video is required to be edited and rendered again, and time and resources are very consumed. Second, decoding the target video and then adding a station caption at each image frame is also labor intensive, time consuming and resource consuming. In addition, this requires an image processing algorithm to synthesize the above-described logo image at a specific position, and the video encoder re-encodes the image frame to which the logo is added and forms the video stream again. This means that the original image (image without icon added) is converted into a logo-added image before the encoder and then compressed, and video stream data is formed, and the compression process exists in the middle, which may affect the quality of food, especially the high-compression processed video. And with this implementation, also depending on specialized hardware or software, additional cost and complexity can be incurred.
Disclosure of Invention
Aiming at the defects of the prior art, the invention aims to provide a method for conveniently adding station marks on an LED display screen, which can solve the problems described in the background art.
The technical scheme for realizing the purpose of the invention is as follows: the method for conveniently adding the station logo on the LED display screen is applied to an LED display screen system, the LED display screen system comprises an LED display screen and a control board for driving the LED display screen to display, fastCGI and a compiled Nginx server are configured on the control board, and the LED display screen is in communication connection with the control board, and comprises the following steps:
Step 1: the Nginx server forwards the received HTTP request from the user to FastCGI, fastCGI running on the address and the port of the control panel to process the HTTP request from the user by establishing an independent process, wherein the process is defined as FastCGI process, the HTTP request comprises a data packet for representing a station logo, the data packet is defined as a station logo data packet, and the station logo data is JSON data;
step 2: analyzing the received station caption data packet through FastCGI, namely analyzing JSON data, and creating a socket connection to perform network interaction with a driving program used for driving the LED display screen on the control panel;
Step 3: after receiving the data packet sent by the FastCGI process, the driver of the control panel analyzes the data part of the actual characterization station caption to obtain the analyzed station caption data;
step 4: loading the characters of the station caption data using a FreeType library preconfigured in the control board, rendering the characters of the station caption data one by one to generate character bitmap data, converting the character bitmap data into pixel data,
Rearranging pixel data based on pixel position offset, wherein the pixel position offset is calculated according to the following formula:
New position of pixel data = original position of pixel data + displacement amount b,
Wherein the displacement amount b= (total width of final bitmap/number of characters) ×current pixel data sequence number;
step 5: and transmitting the final character bitmap data subjected to the position offset to an LED display screen, wherein the LED display screen displays the station logo based on the final character bitmap data.
Further, in step 1, the configuration file of the nmginx protocol is modified on the nmginx server to enable the HTTP request to be forwarded to the FastCGI processes running on the control board address and port for processing.
Further, in step 1, the station caption data packet is derived from the data of the user-defined characterization station caption on the web page, including color, font, character, font length, character length and width attributes, and is assembled in a JSON data to form a JSON format data packet, and the JSON data is sent to the address and port of the designated nginnx server through the HTTP request, and then the FastCGI process processes the HTTP request.
Further, in step 2, in the process FastCGI, the data length of the HTTP request is obtained through the environment variable, then a corresponding memory space is opened up according to the data length, then the data in the HTTP request is read out from the stdin interface of the control board, and the read data is stored in the memory space just opened up.
Further, in step 2, the method further includes: in FastCGI processes and control panel drivers, custom data encapsulation formats are predefined.
Further, in step 5, the control board is preconfigured with an FPGA circuit board, and generates a BRAM through Block RAM Generator IP cores in Vivado of the FPGA circuit board, and first writes the final character bitmap data into the BRAM, so as to store the final character bitmap data in a memory in the FPGA circuit board,
Then, the final character bitmap data is configured using the AXI HDMI TX IP core of Vivado so that the final character bitmap data can be displayed with the resolution and color depth of the supported LED display screen, and the final bitmap data is periodically read from BRAM and sent to the HDMI TX IP core in the FPGA circuit board to be sent to the LED display screen through the HDMI interface.
The beneficial effects of the invention are as follows: compared with the prior art, the method has higher flexibility, can change the station caption at any time according to the requirement, does not need to re-edit and render the video, can realize real-time update, and can be displayed on the LED display screen only by submitting a new station caption. Because a large amount of computing resources are not needed to re-render the video, the resource consumption is smaller and the efficiency is higher.
Drawings
FIG. 1 is a schematic flow chart of the present invention.
Detailed Description
The invention is further described with reference to the accompanying drawings and detailed description below:
As shown in fig. 1, a method for adding station marks on an LED display screen is applied to an LED display screen system, the LED display screen system includes an LED display screen and a control board for driving the LED display screen to display, fastCGI (universal gateway interface) and a compiled nginnx server (high-performance HTTP and reverse proxy web server) are configured on the control board, and the LED display screen is in communication connection with the control board, and includes the following steps:
step 1: the Nginx server forwards the received HTTP request from the user to FastCGI, fastCGI running on the address and port of the control board to process the HTTP request from the user by establishing an independent process, wherein the HTTP request comprises a data packet representing a station logo, which is defined as a station logo data packet, and the station logo data is JSON data.
FastCGI is a protocol commonly used for dynamic Web services that allows tasks to be separated from the Web server process that handle client requests, and handed to a separate set of FastCGI processes for processing.
The configuration file of the nmginx protocol is modified on the nmginx server to enable HTTP requests to be forwarded to FastCGI processes running on the control board address and port for processing, the process established by FastCGI being defined as the FastCGI process. The nmginx server refers to a server configured with an nmginx protocol.
The user can customize the data representing the station caption on the webpage and form a data packet, wherein the data representing the station caption comprises attributes such as color, font, characters, font length, character offset length and width, and the like, and the data are assembled in one JSON data, so that the data packet in the JSON format is formed. The JSON data is sent to the specified server (i.e., nmginx server) address and port via HTTP requests, which are then processed by FastCGI processes.
Step 2: and analyzing the received station caption data packet through FastCGI, namely analyzing JSON data, and creating a socket connection to perform network interaction with a driving program used for driving the LED display screen on the control panel.
In FastCGI processes, the data length of the HTTP request is obtained through the environment variable, and then a corresponding memory space is opened up according to the data length. Then, the data in the HTTP request is read out from the stdin interface (standard input) of the control board, and the read data is stored in the memory space just opened.
In an alternative embodiment, in the network interaction process, in order to ensure that data can be correctly sent and received, accuracy of data transmission is improved. In FastCGI processes and control panel drivers, custom data encapsulation formats are predefined. In this example, the custom data encapsulation format begins with byte sequences 0x42, 0x4C, 0x08, 0x35, followed by two byte lengths to represent the data length, and a small end endian (i.e., low order, high order). If the data length is less than 256, the high byte is 0. The user requested data itself, whose length is determined by the first two bytes, ends with byte 0x00, to identify the termination of the packet.
In addition, in order to ensure that a driver on the control board can receive the parsed station caption data packet, the control board returns a section of byte sequence to the FastCGI process after receiving the station caption data packet, and the FastCGI process starts a thread, which is specially used for processing timeout and receiving data from control return.
The FastCGI process also feeds back whether the parsed station caption data packet is successfully sent to the user (the client where the user is located) through the Nginx server, and presents the result of successful sending of the overtime or custom station caption to the client. And closing the socket connection after the processing is finished.
In this process, fastCGI processes mainly play a role of an intermediary, package the web page POST request (i.e. HTTP request), and add a certain protocol to prevent data from sticking to the package during socket connection transmission, and have a certain encryption effect. Then FastCGI processes forward the data to the control board, receive the information returned by the control board, and feed back to the client. The actual data processing steps are performed within the control panel program.
Step 3: after receiving the data packet sent by FastCGI processes, the driver of the control board firstly checks the head identification of the data packet, then analyzes the data part of the actual characterization station logo according to the data length, and finally confirms whether the data packet is complete or not, namely, the data packet is completely received according to the end identification. Analyzing the processed data packet representing the station caption means obtaining the data of the attribute of the representing station caption, such as color, font, character, font length, character offset length and width, and the like, and obtaining the analyzed station caption data. The method also comprises the step of storing the data part of the analysis and processing characterization station caption.
Step 4: the characters of the station caption data are loaded by using a FreeType library which is pre-configured in the control panel, the font size, width and height of the station caption data are set by using the FreeType library, and the fonts of the station caption data are rendered one by using the FT_Load_Char function of the FreeType library, so that character bitmap data are generated.
Wherein, the character bitmap data can be stored in a temporary variable bitmap.
Next, the character bitmap data is converted into hexadecimal pixel data and saved into a file.
After the character bitmap data is converted into pixel data, a linear array is obtained, so that pixels based on the offline array can be rendered in a specified range of the LED screen.
In order to meet the display requirement of the display screen, the station caption is usually displayed at a specific position, such as the upper right corner or the upper left corner. This requires that the characters be rendered within a specified range, i.e. that the pixel data need to be rearranged. The rearrangement of the pixel data is mainly based on the number of characters, the sequence of the characters and the position of the pixels of each character in the original bitmap, and the rearrangement of the pixel data is determined based on the pixel position offset, so that the final character bitmap data is obtained. In other words, each automatic pixel needs to be positioned so that the pixels of the character are evenly distributed in the final bitmap as a station caption.
The position offset of the pixel can be calculated as follows: new position of pixel data = original position of pixel data + displacement b, where displacement b= (total width of final bitmap/number of characters) × current pixel data sequence number.
The original position of the pixel data refers to the position of the current pixel in the original bitmap (i.e. the parsed station caption data), the total width of the final bitmap refers to the width of the final bitmap forming the station caption, and the number of characters is the total number of characters in the pixel data. The current pixel data sequence number refers to the sequence number of the current pixel in all the pixel data, and the sequence number starts from 0.
By the above formula, the original position of the pixel data is shifted by a corresponding shift amount, which is dependent on the sequence number of the current pixel data and the total number of characters in the pixel data. In this way, it can be ensured that the pixels of each character can be evenly distributed in the final bitmap, and that the characters can be rendered within a specified range. In other words, the formula will determine whether each pixel is within the rendering region of the character, and if so, its pixel value will be greater than 0, i.e., considered to be within the rendering region of the character, and the pixel value is the bitmap data for the character. If the pixel value is less than or equal to 0, the pixel value is set to 0.
Step 5: and writing the final character bitmap data subjected to the position offset into a file and storing the final character bitmap data, and sending the final character bitmap data to an LED display screen, wherein the LED display screen displays the station logo based on the final character bitmap data.
In this step, the control board is preconfigured with the FPGA circuit board, and generates a BRAM through Block RAM Generator IP cores in Vivado of the FPGA circuit board, and the final character bitmap data is written into the BRAM first, so as to be stored in the memory in the FPGA circuit board. The final character bitmap data is then configured using the AXI HDMITX IP cores of Vivado to enable the final character bitmap data to be displayed at the resolution and color depth of the supported LED display screen. And periodically reading the final bitmap data from the BRAM, and sending the final bitmap data to an HDMI TX IP core in the FPGA circuit board so as to be sent to the LED display screen through an HDMI interface.
The invention can enable the interface of the self-defined station caption and the data structure generated by the interface to be: the user can customize the station caption through the webpage, including various aspects such as fonts, colors, characters, font length, character offset width length and the like, and after the user-defined settings are assembled, a JSON data structure is formed; specific protocol for data interaction between FastCGI processes and control boards: the data interaction between FastCGI processes and the control panel adopts a specific data encapsulation format so as to improve the accuracy of data transmission; when each character is rendered, the offset of the character and a specific character offset algorithm are considered, so that the method for generating hexadecimal two-dimensional array lattice data by using the FreeType library can meet specific layout requirements more accurately: the method comprises the whole process of setting the size, width and height of fonts by using a FreeType library, rendering the requested characters one by one, and storing hexadecimal pixel data after conversion of the generated character bitmap data into a file.
Compared with the prior art, the method has higher flexibility, can change the station caption at any time according to the requirement, does not need to re-edit and render the video, can realize real-time update, and can be displayed on the LED display screen only by submitting a new station caption. Because a large amount of computing resources are not needed to re-render the video, the resource consumption is smaller and the efficiency is higher.
The embodiment disclosed in the present specification is merely an illustration of one-sided features of the present invention, and the protection scope of the present invention is not limited to this embodiment, and any other functionally equivalent embodiment falls within the protection scope of the present invention. Various other corresponding changes and modifications will occur to those skilled in the art from the foregoing description and the accompanying drawings, and all such changes and modifications are intended to be included within the scope of the present invention as defined in the appended claims.
Claims (6)
1. The method for conveniently adding the station logo on the LED display screen is characterized by being applied to an LED display screen system, wherein the LED display screen system comprises an LED display screen and a control board for driving the LED display screen to display, fastCGI and a compiled Nginx server are configured on the control board, and the LED display screen is in communication connection with the control board, and comprises the following steps:
Step 1: the Nginx server forwards the received HTTP request from the user to FastCGI, fastCGI running on the address and the port of the control panel to process the HTTP request from the user by establishing an independent process, wherein the process is defined as FastCGI process, the HTTP request comprises a data packet for representing a station logo, the data packet is defined as a station logo data packet, and the station logo data is JSON data;
step 2: analyzing the received station caption data packet through FastCGI, namely analyzing JSON data, and creating a socket connection to perform network interaction with a driving program used for driving the LED display screen on the control panel;
Step 3: after receiving the data packet sent by the FastCGI process, the driver of the control panel analyzes the data part of the actual characterization station caption to obtain the analyzed station caption data;
step 4: loading the characters of the station caption data using a FreeType library preconfigured in the control board, rendering the characters of the station caption data one by one to generate character bitmap data, converting the character bitmap data into pixel data,
Rearranging pixel data based on pixel position offset, wherein the pixel position offset is calculated according to the following formula:
New position of pixel data = original position of pixel data + displacement amount b,
Wherein the displacement amount b= (total width of final bitmap/number of characters) ×current pixel data sequence number;
step 5: and transmitting the final character bitmap data subjected to the position offset to an LED display screen, wherein the LED display screen displays the station logo based on the final character bitmap data.
2. The method for adding station logos to an LED display screen according to claim 1, wherein in step 1, the configuration file of the nmginx protocol is modified on the nmginx server to enable HTTP requests to be forwarded to FastCGI processes running on the control board address and port for processing.
3. The method for adding station marks conveniently to an LED display screen according to claim 1, wherein in step 1, the station mark data packet is derived from the data of the user-defined characterization station mark on the web page, including color, font, character, font length, character length and width attributes, and is assembled in a JSON data packet to form a JSON format, and the JSON data packet is sent to the address and port of the designated nginnx server through an HTTP request, and then the FastCGI processes the HTTP request.
4. The method for adding station marks to the LED display screen conveniently according to claim 1, wherein in step 2, in the FastCGI process, the data length of the HTTP request is obtained through the environment variable, then a corresponding memory space is opened up according to the data length, then the data in the HTTP request is read out from the stdin interface of the control board, and the read data is stored in the memory space just opened up.
5. The method for adding station logos to an LED display screen according to claim 1, further comprising, in step 2: in FastCGI processes and control panel drivers, custom data encapsulation formats are predefined.
6. The method for adding station marks conveniently to an LED display screen according to claim 1, wherein in step 5, the control board is pre-configured with an FPGA circuit board, and generates a BRAM through Block RAM Generator IP cores in Vivado of the FPGA circuit board, and the final character bitmap data is written into the BRAM first, so as to be stored in a memory in the FPGA circuit board,
Then, the final character bitmap data is configured using the AXI HDMI TX IP core of Vivado so that the final character bitmap data can be displayed with the resolution and color depth of the supported LED display screen, and the final bitmap data is periodically read from BRAM and sent to the HDMI TX IP core in the FPGA circuit board to be sent to the LED display screen through the HDMI interface.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311478324.1A CN117354442B (en) | 2023-11-07 | 2023-11-07 | Method for conveniently adding station logo on LED display screen |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311478324.1A CN117354442B (en) | 2023-11-07 | 2023-11-07 | Method for conveniently adding station logo on LED display screen |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117354442A CN117354442A (en) | 2024-01-05 |
| CN117354442B true CN117354442B (en) | 2024-06-14 |
Family
ID=89369261
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311478324.1A Active CN117354442B (en) | 2023-11-07 | 2023-11-07 | Method for conveniently adding station logo on LED display screen |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117354442B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107527377A (en) * | 2017-08-25 | 2017-12-29 | 扆冰蕾 | A kind of hanging Chinese character rendering intent of three-dimensional earth based on mobile platform |
| CN110287043A (en) * | 2019-07-01 | 2019-09-27 | 山东浪潮人工智能研究院有限公司 | A kind of AWG control method and AWG control system |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6993594B2 (en) * | 2001-04-19 | 2006-01-31 | Steven Schneider | Method, product, and apparatus for requesting a resource from an identifier having a character image |
| US7565402B2 (en) * | 2002-01-05 | 2009-07-21 | Eric Schneider | Sitemap access method, product, and apparatus |
| US20210377330A1 (en) * | 2018-10-22 | 2021-12-02 | Radiant Communications Corporation | Low-latency video internet streaming for management and transmission of multiple data streams |
| CN112040168A (en) * | 2020-09-04 | 2020-12-04 | 苏州科达科技股份有限公司 | Station caption processing method, electronic device and storage medium |
-
2023
- 2023-11-07 CN CN202311478324.1A patent/CN117354442B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107527377A (en) * | 2017-08-25 | 2017-12-29 | 扆冰蕾 | A kind of hanging Chinese character rendering intent of three-dimensional earth based on mobile platform |
| CN110287043A (en) * | 2019-07-01 | 2019-09-27 | 山东浪潮人工智能研究院有限公司 | A kind of AWG control method and AWG control system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117354442A (en) | 2024-01-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111899322B (en) | Video processing method, animation rendering SDK, equipment and computer storage medium | |
| CN110996160B (en) | Video processing method and device, electronic equipment and computer readable storage medium | |
| US20080222504A1 (en) | Script-based system to perform dynamic updates to rich media content and services | |
| CN110784730A (en) | Live video data transmission method, device, equipment and storage medium | |
| US8213732B2 (en) | Device and method for simplifying vector graphics | |
| US20090028428A1 (en) | Media Content Management | |
| US20240323544A1 (en) | Image capture method, image display method, and apparatus | |
| KR20090025222A (en) | Text Remoting Method | |
| JP2024513380A (en) | Subtitle processing methods, devices, devices and storage media | |
| CN117354442B (en) | Method for conveniently adding station logo on LED display screen | |
| CN119626148A (en) | A direct drive display method for LED display screen without sending card and receiving card | |
| CN113315982B (en) | Live broadcast method, computer storage medium and equipment | |
| CN115589408B (en) | System and synchronization method for multi-terminal online PPT playing, electronic device and storage medium | |
| EP3744105B1 (en) | Multimedia communication bridge | |
| CN111064986B (en) | Animation data sending method with transparency, animation data playing method and computer equipment | |
| JP2010220186A (en) | Video converter, video output device, video conversion system, video image, recording medium and video conversion method | |
| CN115150390A (en) | Image display method, device, equipment and medium | |
| CN115292426A (en) | Map access method, device, server and storage medium | |
| CN112632902A (en) | Text processing method and device, text playing method and device and text playing control system | |
| CN114970449B (en) | Font processing method applied to on-screen communication | |
| CN114245188B (en) | Automatic cascade method of playing box terminal and cascade playing method of LED display screen | |
| US8639845B2 (en) | Method for editing multimedia pages on a terminal using pre-stored parameters of objects appearing in scenes | |
| US20250005778A1 (en) | Fractal-geometry based fingerprint value determination from media content | |
| KR100374021B1 (en) | A display driver for an Internet TV and a method for driving a display device of an Internet TV | |
| CN120047576A (en) | Method and system for displaying texts of multiple display devices |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |