WO2024144523A1

WO2024144523A1 - Providing 3d imaging on a platform with a real-time operating system

Info

Publication number: WO2024144523A1
Application number: PCT/TR2023/051007
Authority: WO
Inventors: Fikret OZTURK; Mehmet Yildirim; Gokhan Sahin; Mesut PAK; Hasan Feyzi DOGRUYOL
Original assignee: Havelsan Hava Elektronik Sanayi ve Ticaret AS
Current assignee: Havelsan Hava Elektronik Sanayi ve Ticaret AS
Priority date: 2022-12-28
Filing date: 2023-09-21
Publication date: 2024-07-04
Anticipated expiration: 2025-06-28

Abstract

The invention relates to a method for displaying the graphical interface on the screen on platforms with a Graphics Processing Unit (GPU)ZDisplay Controller (DC) electronic hardware and a real-time operating system. In the inventive method, the Mesa3D library enables all graphics operations to be performed on the CPU and the graphics to be displayed on the screen with the Online-Screen and the Screen Driver layers integrated into the Mesa3D library.

Description

PROVIDING 3D IMAGING ON A PLATFORM WITH A REAL-TIME OPERATING SYSTEM

Technical Field

The invention relates to a method for displaying the graphical interface on the screen on platforms with a Graphics Processing Unit (GPU)/Display Controller (DC) electronic hardware and a real-time operating system. In the inventive method, the Mesa3D library enables all graphics operations to be performed on the CPU and the graphics to be displayed on the screen with the Online-Screen and the Screen Driver layers integrated into the Mesa3D library.

Prior Art

On current real-time operating system platforms, GPU and/or CPU-based 3D drawing operations are performed. GPU-based drawing requires different windowing infrastructure and GPU driver software that implemented for different real-time operating systems. In CPU-based 3D drawing operations, the generated scene is transmitted to the screen with a library that provides the windowing infrastructure.

In GPU-based applications, 3D graphics hardware-rendering requires both GPU hardware and operating system-specific display drivers. In current real-time operating systems, there is no general display driver software to address all GPUs. In current real-time operating systems, different display driver software needs to be implemented for each GPU.

In CPU-based applications, 3D graphics -rendering requires the use of a library that provides a windowing infrastructure dependent on the real-time operating system. This prevents 3D graphics drawing on real-time operating systems that do not have a windowing system infrastructure. In the known state of the art, the International Patent document WO2020135721 Al mentions a method and system developed to implement the 3D capability of a cloud desktop virtual machine.

In the known state of the art, the Chinese Patent CN114116227A mentions an imaging method, device and equipment based on the Wayland protocol without GPU support.

When the methods available in the art were examined, it was necessary to realize the method of the invention in which all graphics operations are performed on the CPU with the Mesa3D library and the graphics are displayed on the screen with the Online-Screen and Screen Driver layers integrated into the Mesa3D library.

Objectives of the Invention

The object of the present invention is to realize a method in which all graphics operations are performed on the CPU with the Mesa3D library and the graphics are displayed on the screen with the Online-Screen and Screen Driver layers integrated into the Mesa3D library.

Another object of the present invention is to provide a method for performing CPUbased software graphics drawing operations on real-time operating systems.

Another object of the present invention is to realize a method of displaying the screen image using all graphics processing units independent of the hardware manufacturer with display driver software designed for real-time operating systems.

Another object of the present invention is to realize a method that eliminates the need for different display drivers for different hardware units and accordingly accelerates the project development process.

Detailed Description of the Invention An exemplary embodiment of the method for achieving the objects of the present invention is shown in the attached figures. These figures;

Figure 1: This is a schematic view of an exemplary embodiment of the inventive method.

Figure 2: This is a view of the flowchart of the inventive method.

The parts in the figures are numbered one by one and the equivalents of these numbers are given below.

1. Real-Time Operating System (RTOS)

2. User Space

3. Kernel Space

4. Hardware Layer

5. OpenGL Library

6. Mesa3D Graphics Library

7. State Tracker

8. Gallium3D Layer

9. Softpipe Driver Layer

10. Winsys Layer

11. Online Screen Module

12. Display Driver Module

13. S oftware Rendering

14. Real-time Operating System (RTOS) Layer

15. Graphics Processor Unit (GPU)/Display Controller (DC)

The invention relates to a method for displaying a graphical interface on a screen on platforms having a Graphics Processing Unit (GPU)/Display Controller (DC) (15) electronic hardware and a real-time operating system (1), and includes the steps, - Making the Graphics Processor Unit (GPU)ZDisplay Controller (DC) (15) device available (100) for the real-time operating system (1),

- Defining the screen resolution and color format to be displayed (110), Transmitting the desired resolution and color format information to the display driver module (12) via a driver function call (120) within the realtime operating system (1),

- Mapping (130) the address of the memory region allocated in the physical memory space to handle the drawing area to an address in the user space (2) by a driver function call in the real-time operating system (1),

Setting (140) the mapped user space (2) drawing area address in the Gallium 3D layer (8) and Softpipe driver layer (9) to the OpenGL library (5) drawing area with the resolution, color format and the address to be drawn in the user space,

Invoking the graphics drawing calls to the Gallium 3D layer (8) and Softpipe driver layer (9) to draw graphics in the drawing area in user space (150).

In the method of the invention, while realizing the above steps;

At the time of initialization of the real-time operating system (1), defining the control functions that enable the display driver module (12) to be used from the user space (2) and storing them in the system devices (160), Operation of user space (2) by the display device (170),

The Display Driver module (12) allocates the memory area to form the frame buffer according to the resolution information in the physical memory area of the Graphics Processor Unit (GPU)/Display Controller (DC) (15) device with the real-time operating system’ s (1) dynamic memory allocation function (180) steps are also performed.

Real-time operating system (RTOS) (1); is an operating system that guarantees a capability or the ability to respond to external events within a given time constraint. The correctness of the system's behavior depends on the results of the calculations and the time in which these results are produced. Key characteristics for an RTOS (1); long-term operability, supported standards, performance, memory consumption, modularity, suitability for different scale projects. The real-time operating system (1) in the inventive method, in which the Mesa3D (6) OpenGL library (5) is integrated with the software rendering (13) method; is a non-open source operating system that adheres to the POSIX standard and fully supports PSE profiles 51-52-53. Conformance to the POSIX standard is fully verified by the official POSIX conformance test suite. It is a real-time operating system developed specifically for real-time safety critical avionics applications. However, it is also scalable and adaptable for embedded real-time applications of other application areas. Depends on ARINC 653 specification for avionics applications, fully supports services and features required by ARINC 653 Part-1 such as multi -module scheduling, Part-2’ s optional services such as service access points. It has also been developed to be adaptable to the different processor architectures and hardware configurations. Memory manager, device manager modules and hardware abstraction layers such as architecture support package (ASP) and board support package (BSP) have layered structures so that they can be easily ported to many platforms by adapting them to different architectures and hardware configurations.

User space (2) is a conceptual space that contains the virtual memory partition(s) where user applications are executed.

Kernel space (3) is the space allocated to run device drivers, the operating system kernel and all other kernel extensions.

Graphics Processor Unit (GPU)/Display Controller (DC) (15) is the hardware layer consisting of the electronics hardware and the display.

The hardware layer (4) is the layer containing the Graphics Processor Unit (GPU)/Display Controller (DC) (15) and display units. The OpenGL library (5) is a graphics application-development interface for drawing both two- and three-dimensional graphics on the screen, either by using advanced hardware support or by software rendering. Being a standard, OpenGL is widely used in various applications and supported in many operating systems.

Mesa3D graphics library (6) is an open source implementation of graphics API features such as OpenGL library (5), OpenGL ES, and Vulkan to be able to create 3D graphics. It is first developed for and used on Linux systems. The point where the inventive method differs from existing applications is the characteristics of the real-time operating system (1) platform on which the Mesa3D graphics library (6) is ported and also the method of porting contains innovations.

State Tracker (7) is the part that connects the Mesa3D graphics library (6) to the Gallium3D layer (8). It converts any graphics processing operation that is not directly supported by the graphics processing unit into a hardware-friendly form. Receives requests from the application program by Mesa Main and manages the corresponding Rasterizer.

The Gallium3D layer (8) is a set of interfaces and supporting device driver software intended to facilitate the implementation of device drivers for different operating systems. It is available in open source as part of the Mesa3D graphics library (6). It has been used to simplify driver development process and to consolidate the duplicated code of several different drivers into a single common place.

The Softpipe driver layer (9) is a software i.e., not GPU based implementation of the Gallium3D layer (8). When the system does not have a hardware driver, it is used to perform graphics operations on the CPU (Central Process Unit).

The Winsys layer (10) and the online screen module (11) are used to enable drawing on the user-allocated memory without any windowing system or operating system dependencies. Software rendering (13) is a method of rendering images i.e., calculating 2D points corresponding to 3D points using computer software. In graphics rendering, it refers to a rendering process that does not depend on hardware units such as a graphics card. The processing takes place entirely in the CPU (Central Process Unit), independent of the GPU (Graphics Processing Unit). In the method of the invention, this method is used to perform graph drawing on a real-time operating system (1) independently of the graphics processing unit, which is an external hardware.

The Real-Time Operating System (RTOS) Layer (14) consists of the online screen module (11) and the display driver module (12). The display driver module (12) enables drawing for all GPUs, regardless of GPU type.

The points where the inventive method differs from the known systems are that the operating system to which the Mesa3D graphics library (6) and OpenGL library (5) are ported has the feature of a real-time operating system (1), and the graphics drawing requirement of the real-time operating system (1) can be fulfilled independently of external graphics units with the CPU-based software rendering (13) method.

The method of the invention contains a display driver module (12) and an online screen module (11) that can operate independently of GPU/Display controller (15) modulesin real-time operating systems (1). The display driver module (12) provides the screen display for real-time operating systems (1) by using all graphics processing units independent of the hardware manufacturer. The online screen module (11), on the other hand, communicates with the display driver module (12) without any windowing system then sends the drawings realized as a result of OpenGL library (5) calls to the screen.

Furthermore, in the method of the invention, the Linux DRM structure is removed from the Mesa3D graphics library (6).

The windowing libraries in the Windowing system module were removed from the Mesa3D graphics library (6) and replaced with the online screen module (11). A display driver module (12) layer is added to the online screen module (11) layer to respond on the kernel space (3) side. Model independent display driver is provided with GPU/Image controller (15) module. Online screen module (11) provides 3D drawing environment without windowing system. Screen display can be provided without a graphics processing unit, external GPU hardware is not needed then the cost of the application projects will be reduced. Moreover, the labor cost of documentation required for certification has been reduced.

Higher performance levels than the hardware graphics processing unit for display applications with low resolution screens.

The invention method can be used in flight control systems, navigation systems, electronic warfare systems, remote-controlled weapon systems, automotive and rail transportation systems.

Claims

1. The invention relates to a method for displaying a graphical interface on a screen on platforms having a Graphics Processing Unit (GPU)/Display Controller (DC) (15) electronic hardware and a real-time operating system (1), characterized by comprising the steps,

- Making the Graphics Processor Unit (GPU)/Display Controller (DC) (15) device available (100) for the real-time operating system (1),

- Defining the screen resolution and color format to be displayed (HO),

Transmitting the desired resolution and color format information to the display driver module (12) via a driver function call (120) within the real-time operating system (1),

2. The invention relates to a method according to claim 1, characterized in that while performing the method;

At the time of initialization of the real-time operating system (1), the control functions that enable the display driver module (12) to be used from the user space (2) are defined and stored in the system devices (160),

Activation of the user space (2) to use the display device (170), The Display Driver module (12) allocates the memory area to form the frame buffer according to the resolution information in the physical memory area of the Graphics Processor Unit (GPU)/Display Controller (DC) (15) device with the real-time operating system’s (1) dynamic memory allocation function (180) steps are also performed.