DE102022111182A1 - METHOD AND DEVICE FOR CONTROLLING A COLLECTION OF VEHICLE ENVIRONMENTAL DATA COLLECTED BY MEANS OF A SENSOR SYSTEM BUILT ON A VEHICLE - Google Patents

Abstract

A method is provided for controlling the collection of vehicle surroundings data, which are collected by means of a sensor system installed on a vehicle, the method comprising: determining a specification as to which type of vehicle surroundings data is to be collected from the vehicle at which position, based based on already collected vehicle environmental data and/or a user specification in a first cloud component, and outputting the specification from the first cloud component to the vehicle, so that the vehicle can control the collection of vehicle environmental data based on the received specification and its current position.

Description

The present disclosure relates to a method for controlling the collection of vehicle surroundings data, which is collected using a sensor system installed on a vehicle. Additionally or alternatively, a system for data processing or a data processing device is provided, which is designed to at least partially carry out the method. Additionally or alternatively, a motor vehicle, in particular an automated one, is provided with the data processing device. Additionally or alternatively, a computer program is provided which includes commands which, when the program is executed by a computer, cause it to at least partially carry out the method. Additionally or alternatively, a computer-readable medium is provided which includes instructions which, when the instructions are executed by a computer, cause it to at least partially carry out the method.

Vehicle Sensor Data (VSD) or vehicle environment data is a typical use case for obtaining large-scale sensor data from a vehicle fleet relating to an environment or an environment of a vehicle from various sensors (e.g. odometry, GPS, camera, radar, lidar, ultrasound, etc ) to collect and transfer to a cloud.

A typical use of the VSD can be found in map generation. In the vehicle, the collected sensor data is usually pre-aggregated and compressed in order to reduce the amount of data to be transmitted.

There is also cloud-controlled campaign management or fleet control, which can be used to activate or deactivate entire fleet parts. Possible parameters include markets, vehicle models, software versions and/or larger geographical regions such as countries.

In this context the describes WO 2021/042051 A1 an end-to-end system for data generation, map generation using the generated data and localization regarding the generated map. Map streams - or streams of sensor data, perceptual output from deep neural networks (DNNs), and/or relative trajectory data - corresponding to any number of trips by any number of vehicles can be generated and uploaded to a cloud. The map streams can be used to generate map data - and ultimately a merged high-definition (HD) map - representing data generated over a variety of trips. When localizing on the fused HD map, individual localization results can be generated based on comparisons of real-time data from a sensor modality with map data from the same sensor modality. This process can be repeated for any number of sensor modalities and the results can be fused together to produce a final fused localization result.

With the existing campaign management systems and expansion levels, certain vehicle classes can be activated/deactivated separately, but all vehicles of a class and geographical region are activated/deactivated. In addition, all activated vehicles in a class collect the same sensor data, i.e. all vehicles collect all available data or always the same subset.

The query as to which data should be collected is sent by the vehicle to the cloud at the beginning of the journey (so-called waking up of the vehicle electronics) and is only updated during the journey if the data protection settings are changed by a driver or user of the vehicle.

Against the background of this prior art, the task of the present disclosure is to specify a device and a method which are each suitable for at least enriching the prior art.

The task is solved by the features of the independent claim. The subclaims contain preferred developments of the invention.

The task is then solved by a method for controlling the collection of vehicle surroundings data, which is collected using a sensor system installed on a vehicle.

The method can be a computer-implemented method, i.e. one or more or all steps of the method can be carried out at least partially using a computer or a data processing system.

The process enables intelligent fleet control for VSD data collection.

The method includes setting a specification as to what type of vehicle environment data is to be collected from the vehicle at which position, based on vehicle data that has already been collected. Environmental data and/or a user specification in a first cloud component.

The method includes outputting the specification from the first cloud component to the vehicle so that the vehicle can control the collection of vehicle surroundings data based on the received specification and its current position.

In other words, and based on a specific embodiment, the method can be used to provide a detailed and dynamic specification from the cloud compared to the prior art described at the beginning as to which data is to be collected on which road sections. This target can be calculated based on the data collected from the entire fleet in the cloud. Each individual vehicle can then receive a specific assignment from the cloud via an interface based on its exact position (e.g. through GPS and navigation map and localization) as to which data should be collected and ultimately sent to the cloud. In addition, each vehicle can receive updates while driving as to whether the data collection strategy needs to be adjusted.

The default may be a software component that includes an algorithm that controls the collection of environmental data.

With the method described above, in comparison to the prior art described at the beginning, “finer” control parameters regarding the VSD to be collected can be sent to the vehicle, so that intelligent control of the vehicle fleet takes place in order to only provide the required data at certain positions or in certain areas, e.g. over certain road sections, and thus reduce the data bandwidth and transfer and process costs.

A solution is therefore proposed as to how the data collection in the VSD can be controlled in a much more fine-grained manner and thus the challenges mentioned above with reference to the state of the art can be solved. One goal is to reduce the amount of data that has to be transferred from the vehicle to the cloud, which results in a cost reduction for data transfer and data processing.

Possible further developments of the methods described above are explained in detail below.

Setting the default may include algorithmically determining the default based on the already collected vehicle environmental data and/or the user default. In other words, the method can include an algorithmic adjustment of the campaign parameters, taking into account the collected data and the user's specified target variables (e.g.: coverage of a road section).

The setting and output of the specification can be carried out continuously, so that the method includes a continuous adjustment of the specification based on a current user specification and/or vehicle environmental data collected in the meantime. By vehicle environmental data collected in the meantime, it is meant that such vehicle environmental data is taken into account that is collected after the (first or previous) setting and before the adjustment of the (first or previous) specified specification in order to produce an updated or second one through the adjustment to obtain the specification. This can also be referred to as dynamic adaptation, as already indicated above.

The method can include providing a digital map comprising information about the already collected vehicle surroundings data via a programming interface of the first cloud component, receiving a plurality of user requests via the programming interface, each comprising information at which position of the digital map and what type of vehicle surroundings data from the vehicle to be collected, and determining the user preference based on the multiple user requirements.

The method may include adjusting a general data collection strategy determined by a second cloud component for a vehicle fleet to which the vehicle belongs based on the specification determined by the first cloud component.

Setting the default may further be based on information received from the second cloud component that determines the general data collection strategy for the vehicle fleet to which the vehicle belongs.

The method may include controlling the collection of vehicle surroundings data based on the specification set by the first cloud component using the vehicle. The method may include sending or outputting the collected vehicle environment data from the vehicle to the first and/or the second cloud component.

What has been described above can be summarized in other words and in a concrete form related, which is described as not limiting the present disclosure, can be summarized as follows: A VSD-specific cloud component, the so-called first cloud component, with a user API (application programming interface), the so-called programming interface, be provided. The VSD-specific cloud component can specify the desired data collection strategy (sensor & data features, geo-position, time period, etc.). Different users can define their requirements regarding the data collection strategy via the user API provided by the VSD-specific cloud component. The VSD Cloud component can then determine the data collection strategy that meets all requirements while minimizing the amount of data. The data collection strategy can be communicated to the general campaign management component of the cloud and/or directly to the vehicle fleet. The general campaign management component is used to control data collection activities based on static parameters such as market, vehicle class, equipment, and individual data protection requirements. An on-board vehicle component and logic can enable precise vehicle localization on a map, for example using GPS and other sensors, and request a vehicle-specific, event- and/or time-controlled data collection strategy with these dynamic parameters. This makes it possible to dynamically adapt data collection to meet the specified data collection strategy. More precisely, in the VSD-specific cloud component, requests can be received from any user using the User API. The user can specify their requirements based on the provided (navigation/SD) card. User requirements can be stored for each road section in the map, e.g. number of crossings, time period, required features, etc. A VSD strategy component, which is part of the VSD-specific cloud component, can be used to determine a strategy that covers the requirements of all users . On the one hand, this global data collection strategy can be communicated directly to the campaign management component of the cloud in order to control general campaign management. On the other hand, the vehicle-specific strategy can be communicated directly to any vehicle that requests the strategy. Campaign management can represent the general activation/deactivation of vehicles, e.g. to meet global VSD requirements or to meet data protection and other legal requirements. Each vehicle can request the strategy from VSD Service at the start of the journey, and event and/or time-controlled during the journey. The relevant strategy for each vehicle can be determined based on its position, time and/or equipment and the data and targets collected from the fleet. The vehicle position can be used for the most precise referencing possible to map links or road sections. Each vehicle can adapt the VSD data collection to the strategy received and only send the desired data to the cloud. In particular, only the data from the fleet that is required for the use cases is then collected. This allows the costs for data transmission and processing costs to be reduced.

Furthermore, a system for data processing is provided. The system is characterized in that it is designed to at least partially carry out the method described above.

The system may include a data processing device in a vehicle. This can be, for example, an electronic control unit or control unit (ECU = electronic control unit). The electronic control unit can be an intelligent processor-controlled unit that can communicate with other modules, for example via a central gateway (CGW) and, if necessary, via field buses such as the CAN bus, LIN bus, MOST bus and FlexRay or via Automotive Ethernet, e.g. together with telematics control devices, can form the vehicle on-board network. It is conceivable that the control unit controls functions relevant to the driving behavior of the motor vehicle, such as the engine control, the power transmission, the braking system and/or the tire pressure monitoring system, and/or can access the relevant data. In addition, driver assistance systems, such as a parking assistant, an adapted cruise control (ACC), a lane keeping assistant, a lane change assistant, a traffic sign recognition, a light signal recognition, a starting assistant, a night vision assistant, an emergency braking assistant and / or an intersection assistant, can be used by the Control unit can be controlled and/or relevant data can be accessed.

Furthermore, a vehicle, a (in particular automated) motor vehicle, can be provided with the data processing device. The vehicle is characterized in that it has the data processing device.

The motor vehicle can be a passenger car, in particular an automobile. The automated motor vehicle can be designed to at least partially and/or at least temporarily take over longitudinal guidance and/or transverse guidance during automated driving of the motor vehicle. The automated one Driving can be done in such a way that the movement of the motor vehicle is (largely) autonomous. The automated driving can be controlled at least partially and/or temporarily by the data processing device.

The motor vehicle can be a motor vehicle with autonomy level 0, i.e. the driver takes over the dynamic driving task, even if supporting systems (e.g. ABS or ESP) are present.

The motor vehicle can be a motor vehicle with autonomy level 1, i.e. have certain driver assistance systems that support the driver in operating the vehicle, such as adaptive cruise control (ACC).

The motor vehicle can be a motor vehicle of autonomy level 2, i.e. be partially automated so that functions such as automatic parking, lane keeping or lateral guidance, general longitudinal guidance, acceleration and/or braking are taken over by driver assistance systems.

The motor vehicle can be a motor vehicle of autonomy level 3, i.e. conditionally automated so that the driver does not have to continuously monitor the vehicle system. The motor vehicle independently carries out functions such as triggering the turn signal, changing lanes and/or keeping in lane. The driver can turn his attention to other things, but if necessary the system will ask him to take over within a warning period.

The motor vehicle can be a motor vehicle with autonomy level 4, i.e. so highly automated that control of the vehicle is permanently taken over by the vehicle system. If the system can no longer handle the driving tasks, the driver can be asked to take over the lead.

The motor vehicle can be a motor vehicle with autonomy level 5, i.e. so fully automated that the driver is not required to fulfill the driving task. No human intervention is required other than setting the target and starting the system.

It is conceivable that the data processing device of the motor vehicle collects the data according to the specification based on an output from a sensor system and outputs it to the first and/or the second cloud component.

A cloud component can be understood as a part of a cloud. A cloud can be understood as meaning a system comprising one or more data processing devices, which is or can be connected to the vehicle, in particular wirelessly.

What has been described above with reference to the procedures also applies analogously to the data processing system and vice versa.

A computer program is also provided. The computer program is characterized in that it includes commands which, when the program is executed by a computer, cause the computer to at least partially carry out at least one of the methods described above.

A program code of the computer program can be in any code, in particular in a code that is suitable for motor vehicle controls.

Furthermore, a computer-readable medium, in particular a computer-readable storage medium, is provided. The computer-readable medium is characterized in that it includes instructions which, when the program is executed by a computer, cause it to at least partially carry out at least one of the methods described above.

That is, a computer-readable medium may be provided that includes a computer program as defined above. The computer-readable medium can be any digital data storage device, such as a USB flash drive, hard drive, CD-ROM, SD card, or SSD card. The computer program does not necessarily have to be stored on such a computer-readable storage medium in order to be made available to the motor vehicle, but can also be obtained externally via the Internet or otherwise.

What has been described above with reference to the methods and system for data processing also applies analogously to the computer program and the computer-readable medium and vice versa.

• 1 zeigt schematisch ein System zur Datenverarbeitung, umfassend Mittel zur Ausführung eines Verfahrens zur Steuerung eines Sammelns von Fahrzeug-Umfelddaten, die mittels einem an einem Fahrzeug verbauten Sensorsystem gesammelt werden, und
• 2 zeigt schematisch ein Ablaufdiagramm des Verfahrens.

An embodiment is described below with reference to 1 and 2 described.

• 1 schematically shows a system for data processing, comprising means for executing a method for controlling the collection of vehicle surroundings data, which are collected by means of a sensor system installed on a vehicle, and
• 2 shows a schematic flowchart of the process.

This in 1 The system 1 shown for data processing has a first cloud component 2, a second cloud component 3 and a vehicle 4, in this case an automobile.

The first cloud component 2 has a digital card 21, which can also be referred to as a reference card, a programming interface 22, a strategy module 23, and a communication interface 24.

The second cloud component 3 has a strategy module 31 and a communication interface 32.

The vehicle 4 has a digital map 41 that corresponds to the reference map, a localization module 42 (e.g. comprising a position sensor, such as GPS), a strategy module 43, a communication interface 44 and a sensor system 45. The sensor system 45 may include a camera, a radar sensor, a LiDAR sensor and/or an ultrasonic sensor, which are (each) designed to record sensor data regarding an environment of the vehicle 4.

The first cloud component 2 serves to define a VSD data collection strategy, which is adapted at runtime while the vehicle is traveling based on fleet data and dynamic parameters of the vehicle. The second cloud component 3 is used to control data collection activities (e.g. installation of so-called data collection jobs and/or activation/deactivation of data collection) based on static parameters (e.g. market, vehicle classes, versions, data protection requirements) in accordance with a general data collection strategy. The general data collection strategy is only adjusted at the beginning of a journey with the vehicle 4 and when the driver changes data protection settings. The strategy module 43 implements the data collection strategies in the vehicle 4 that it receives from the respective cloud component.

This control of data collection will be discussed in detail below with reference to 2 described, which schematically represents a flowchart of the method for controlling the collection of VSD or vehicle surroundings data, which are collected by means of a sensor system installed on a vehicle 4.

In a first step S1 of the method, a general data collection strategy is loaded, which is received by the strategy module 31 via the communication interface 32 of the second cloud component 3 and the communication interface 44 from the strategy module 43 of the vehicle 4 when the vehicle 4 is started. Additionally or alternatively, an existing (default) data collection strategy that has already been saved in the vehicle can also be loaded, e.g. to circumvent a possible problem of a missing cloud connection. The strategy module 31 of the second cloud component 31 first determines the general data collection strategy for a vehicle fleet to which the vehicle 4 belongs.

In a second step S2 of the method, sensor data from the sensor system 4 is collected under the control of the strategy module 43 of the vehicle 4 and the collected sensor data, in particular the collected environment data, is output via the communication interface 44 of the vehicle 4 to the communication interface 24 of the first cloud component 2.

In a third step S3 of the method, the digital map 21 comprising information about the already collected vehicle environment data (see step S2) is provided via the programming interface 22 of the first cloud component 2. The third step S3 then includes receiving several user requests via the Programming interface 22, wherein each of the user requests includes information about at which position on the digital map 21 what type of vehicle surroundings data is to be collected from the vehicle 4. The digital map 21 and the user requirements are output to the strategy module 23.

In a fourth step S4 of the method, a specification is set as to which type of vehicle environment data is to be collected from the vehicle 4 and at which position by means of the strategy module 23 in the first cloud component 2. The specification is set based on the vehicle that has already been collected -Environmental data (see step S2, both from vehicle 4 and based on all other vehicles in the fleet) and a user specification. For this purpose, the user specification is first determined using the strategy module 23 based on the multiple user requests (see step S3). It is conceivable that the user specification is determined in such a way that all user requirements are met. Setting the default then includes an algorithmic determination of the default based on the already collected vehicle environmental data and the user default. When setting the specification, information, e.g. data protection requirements, is taken into account, which is received from the strategy module 31 of the second cloud component 3 to the strategy module 23 of the first cloud component.

In a fifth step S5 of the method, the specification specified in the fourth step S4 is output.

The specification is sent from the strategy module 23 via the communication interface 24 of the first cloud component 2 to the communication interface 44 of the vehicle 4 and from there passed on to the strategy module 43 of the vehicle 4. Additionally or alternatively, the specification can also be sent via the communication interface 32 of the second cloud 3 to the communication interface 44 of the vehicle 4 and from there passed on to the strategy module 43 of the vehicle 4. It is conceivable that the specification is output to the vehicle 4 triggered by the vehicle 4 while the vehicle 4 is driving or operating. For this purpose, the vehicle 4 can transmit its current position in the digital map 41, which is determined by the strategy module 43 of the vehicle 4 based on an output of the localization module 42 and the digital map 41, to the first cloud component 2 via the communication interfaces 24, 44 and so on to obtain the current VSD data collection strategy or the current specification for the current position of the vehicle 4 (in particular for a predetermined radius around the vehicle 4 and / or one or more route sections in the vicinity of the vehicle 4) from the strategy module 23 of the first cloud component 2 .

Furthermore, in the fifth step S5 of the method, the specification is output to the strategy module 31 of the second cloud component 3, which then adapts the general data collection strategy (see step S1) that is used by the strategy module 31 of the second cloud component 3 for the (entire) vehicle fleet to which the vehicle 4 belongs is determined.

In a sixth step S6 of the method, the collection of the vehicle surroundings data of the sensor system 45 is controlled based on the specification set by the first cloud component 3 using the strategy module 43 of the vehicle 4 and the collected vehicle surroundings data is output via the communication interface 44 of the vehicle 4 to the strategy module 23 of the first cloud component 2 via its communication interface 24.

The steps S3 - S6 of the method are carried out continuously or continuously, i.e. the setting of the default in the first cloud component 3 and the output of the updated or current default is carried out continuously, so that the method involves ongoing adjustment of the default based on a current user specification and / or vehicle environmental data collected in the meantime and the collection of the vehicle environmental data is carried out on the current specification to match the current position of the vehicle 4.

It is conceivable, for example, that the specification is adjusted so that vehicle surroundings data is only collected on certain side roads, since, for example, there is already enough information about the surroundings on the main road. Certain exceptions can apply to the main road in accordance with the specification, i.e. it can be specified, for example, that only certain vehicle environmental data (e.g. stop lines and traffic lights) determined by the specification are collected and sent to the first cloud component 2.

This example illustrates how the amount of data can be reduced using the method described above, since only or only the data that is actually needed there is collected and sent to the first cloud component 2.

Reference symbol list

1

Data processing system

2

first cloud component

21

digital map

22

Programming interface

23

Strategy module

24

Communication interface

3

second cloud component

31

Strategy module

32

Communication interface

4

vehicle

41

digital map

42

Localization module

43

Strategy module

44

Communication interface

45

Sensor system with environmental sensor(s)

QUOTES INCLUDED IN THE DESCRIPTION

This list of documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• WO 2021/042051 A1 [0005]

Claims (10)

Method for controlling the collection of vehicle surroundings data, which is collected by means of a sensor system (45) installed on a vehicle (4), the method comprising: - Determining a specification as to what type of vehicle environmental data is to be collected from the vehicle (4) and at which position, based on already collected vehicle environmental data and/or a user specification in a first cloud component (2), and - Outputting the specification from the first cloud component (2) to the vehicle (4), so that the vehicle (4) can control the collection of vehicle environmental data based on the received specification and its current position. Procedure according to Claim 1 , wherein setting the default includes an algorithmic determination of the default based on the already collected vehicle environmental data and / or the user default. Procedure according to Claim 1 or 2 , wherein the specification and output of the specification is carried out continuously, so that the method includes a continuous adjustment of the specification based on a current user specification and / or vehicle environmental data collected in the meantime. Procedure according to one of the Claims 1 until 3 , wherein the method comprises: - Providing a digital map (21) comprising information about the already collected vehicle environment data via a programming interface (22) of the first cloud component (2), - Receiving several user requests via the programming interface (22), each comprising one piece of information at which position of the digital map (21) what type of vehicle surroundings data is to be collected from the vehicle (4), and - determining the user specification based on the multiple user requirements. Procedure according to one of the Claims 1 until 4 , wherein the method comprises adapting a general data collection strategy determined by a second cloud component (3) for a vehicle fleet to which the vehicle (4) belongs, based on the specification determined by the first cloud component (2). Procedure according to one of the Claims 1 until 5 , wherein the specification is further based on information received from the second cloud component (3), which determines a general data collection strategy for a vehicle fleet to which the vehicle (4) belongs. Procedure according to one of the Claims 1 until 6 , wherein the method includes controlling the collection of vehicle surroundings data based on the specification determined by the first cloud component (2) using the vehicle (4). Data processing system (1), comprising means for carrying out the method according to one of Claims 1 until 7 . Computer program, comprising commands which, when the program is executed by a computer, cause it to carry out the method according to one of the Claims 1 until 7 to carry out. Computer-readable medium comprising instructions which, when executed by a computer, cause it to carry out the method according to one of the Claims 1 until 7 to carry out.

Images