[go: up one dir, main page]

CN118714191A - A method for unified registration, discovery and invocation of multiple types of services - Google Patents

A method for unified registration, discovery and invocation of multiple types of services Download PDF

Info

Publication number
CN118714191A
CN118714191A CN202411189600.7A CN202411189600A CN118714191A CN 118714191 A CN118714191 A CN 118714191A CN 202411189600 A CN202411189600 A CN 202411189600A CN 118714191 A CN118714191 A CN 118714191A
Authority
CN
China
Prior art keywords
service
registration
instance
registration center
information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202411189600.7A
Other languages
Chinese (zh)
Other versions
CN118714191B (en
Inventor
王随涛
徐平伟
张翼
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xi'an Singularity Energy Co ltd
Original Assignee
Xi'an Singularity Energy Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xi'an Singularity Energy Co ltd filed Critical Xi'an Singularity Energy Co ltd
Priority to CN202411189600.7A priority Critical patent/CN118714191B/en
Publication of CN118714191A publication Critical patent/CN118714191A/en
Application granted granted Critical
Publication of CN118714191B publication Critical patent/CN118714191B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/50Network services
    • H04L67/54Presence management, e.g. monitoring or registration for receipt of user log-on information, or the connection status of the users
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/133Protocols for remote procedure calls [RPC]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/50Network services
    • H04L67/51Discovery or management thereof, e.g. service location protocol [SLP] or web services

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Telephonic Communication Services (AREA)

Abstract

The invention relates to the technical field of micro-service registration discovery and service call, in particular to a method for uniformly registering, discovering and calling multi-type services, which comprises the following steps: building a micro-service architecture, comprising a service registry, a registration center and a service gateway; the service registry is used for acquiring service instance information from different clusters and synchronizing the service instance information to the registry; the registration center is used for providing service information of service management, service registration, service discovery and service instance list, and the registration center adopts a service end registration mode to realize the decoupling of the registration center and the service; the service gateway is used for synchronously obtaining a service instance list from the registration center and selecting service instance call; the service registry configures an API of the cluster where the access is based on the cluster where the service deployment is located and correctly synchronizes the service instance information; the invention can realize unified service registration and discovery and service call without deployment and micro-service architecture transformation of different services of different systems.

Description

一种多类型服务统一注册、发现和调用的方法A method for unified registration, discovery and invocation of multiple types of services

技术领域Technical Field

本发明涉及微服务注册发现与服务调用技术领域,尤其涉及一种多类型服务统一注册、发现和调用的方法。The present invention relates to the technical field of microservice registration discovery and service invocation, and in particular to a method for unified registration, discovery and invocation of multiple types of services.

背景技术Background Art

在企业内部IT系统中,往往存在着外购、自研、开源等多种不同类型来源的系统,不同的系统往往使用的技术栈和微服务架构是不一样的, 例如常见的Springcloud、Dubbo或者单体服务架构; 不同的系统的部署架构通常也不相同,例如基于固定主机部署、使用mesos、K8s等分布式资源调度和管理框架; 在这种情况下通常每个项目或者系统都需要维护自己的注册中心和服务负载均衡,使得构建公司统一的跨系统服务调用、服务和API管理、服务监控、链路追踪等变得非常的困难,对分散的各个异构系统服务进行整合或者适配需要投入巨大的成本和面临巨大的风险。In the internal IT systems of enterprises, there are often systems from various sources, such as purchased, self-developed, and open source. Different systems often use different technology stacks and microservice architectures, such as the common Springcloud, Dubbo, or monolithic service architecture; the deployment architectures of different systems are usually different, such as deployment based on fixed hosts, using distributed resource scheduling and management frameworks such as mesos and K8s; in this case, each project or system usually needs to maintain its own registration center and service load balancing, making it very difficult to build a unified cross-system service call, service and API management, service monitoring, link tracking, etc. for the company. Integrating or adapting the scattered heterogeneous system services requires huge costs and faces huge risks.

因此,在不改变各个异构系统的部署和服务架构的前提下,如何实现统一的服务注册发现和调用机制,降低服务的管理运维成本,是一个急需解决的问题。Therefore, without changing the deployment and service architecture of various heterogeneous systems, how to implement a unified service registration, discovery and call mechanism to reduce the management and operation costs of services is an urgent problem to be solved.

发明内容Summary of the invention

本发明的目的在于克服现有技术的缺陷,提供一种多类型服务统一注册、发现和调用的方法。The purpose of the present invention is to overcome the defects of the prior art and provide a method for unified registration, discovery and invocation of multiple types of services.

本发明提供了一种多类型服务统一注册、发现和调用的方法,其特征在于,包括如下步骤:The present invention provides a method for unified registration, discovery and invocation of multiple types of services, characterized in that it includes the following steps:

步骤1、搭建微服务架构,包括服务注册器、注册中心和服务网关;Step 1: Build a microservice architecture, including a service registrar, a registration center, and a service gateway;

服务注册器用于从不同的集群中获取服务实例信息,并同步到注册中心;所述注册中心用于提供服务管理、服务注册、服务发现、服务实例列表的服务信息,注册中心采用服务端注册方式,实现注册中心与服务的解耦;服务网关,用于从注册中心同步得到服务实例列表,选择服务实例调用;The service registrar is used to obtain service instance information from different clusters and synchronize it to the registration center; the registration center is used to provide service information for service management, service registration, service discovery, and service instance list. The registration center adopts the server-side registration method to achieve the decoupling of the registration center and the service; the service gateway is used to synchronize the service instance list from the registration center and select the service instance to call;

步骤2、服务注册器根据服务部署所在集群,配置接入所在集群API并正确同步服务实例信息;Step 2: The service registrar configures the access cluster API and correctly synchronizes the service instance information according to the cluster where the service is deployed;

步骤3、当服务实例发生变化时,其所在集群服务实例信息与从注册中心获取全量的服务实例信息进行比对,根据比对结果更新所述服务实例信息并同步至服务注册器;Step 3: When a service instance changes, the cluster service instance information is compared with the full service instance information obtained from the registration center, and the service instance information is updated according to the comparison result and synchronized to the service registrar;

步骤4、服务注册器获取更新后的服务实例信息并注册到注册中心,使注册中心获得最新的服务实例信息;Step 4: The service registrar obtains the updated service instance information and registers it with the registration center, so that the registration center obtains the latest service instance information;

步骤5、服务网关从注册中心获取最新的服务实例信息,根据服务请求匹配最合适的服务实例信息,发起服务调用,转发给服务请求方。Step 5: The service gateway obtains the latest service instance information from the registration center, matches the most appropriate service instance information according to the service request, initiates a service call, and forwards it to the service requester.

进一步,在步骤4中,服务注册器获取服务实例信息的模式包括:Further, in step 4, the mode in which the service registrar obtains service instance information includes:

1)事件监听模式,如果集群支持服务事件监听,服务注册器会实现一个监听器并注册到集群中,以捕获服务状态变化事件,当捕获到事件时,注册器会获取相关服务实例的最新状态信息;1) Event monitoring mode: If the cluster supports service event monitoring, the service registrar will implement a listener and register it with the cluster to capture service status change events. When an event is captured, the registrar will obtain the latest status information of the relevant service instance;

2)轮询模式,如果集群不支持事件监听,服务注册器会采用轮询模式,定时调用集群的API来获取服务实例的列表和状态信息。2) Polling mode: If the cluster does not support event monitoring, the service registrar will use polling mode to periodically call the cluster API to obtain the list and status information of service instances.

进一步,在步骤4中,在将服务实例信息注册到注册中心后,还包括:Furthermore, in step 4, after registering the service instance information to the registration center, it also includes:

步骤4-1,服务注册器与注册中心同步:服务注册器将获取到的服务实例信息与注册中心中的信息进行比对,识别并处理不同状态的服务实例,当服务实例在集群中存在但注册中心中不存在,将其添加到注册中心,识别为新增实例;当服务实例在集群中不存在但注册中心中存在,将其从注册中心删除,识别为停止实例;当服务实例在集群和注册中心中都存在,不做处理,识别为已有实例;Step 4-1, synchronization of service registrar and registration center: the service registrar compares the obtained service instance information with the information in the registration center, identifies and processes service instances in different states. When a service instance exists in the cluster but not in the registration center, it is added to the registration center and identified as a newly added instance; when a service instance does not exist in the cluster but exists in the registration center, it is deleted from the registration center and identified as a stopped instance; when a service instance exists in both the cluster and the registration center, no processing is performed and it is identified as an existing instance;

步骤4-2,遍历和同步所有服务实例:服务注册器循环遍历所有服务实例,并重复S4-1步骤,直到所有服务实例都处理完毕。Step 4-2, traverse and synchronize all service instances: The service registrar loops through all service instances and repeats step S4-1 until all service instances are processed.

进一步,在步骤5中,服务网关从注册中心获取最新的服务实例信息,根据服务请求匹配最合适的服务实例信息,发起服务调用,转发给服务请求方,具体包括:Further, in step 5, the service gateway obtains the latest service instance information from the registration center, matches the most appropriate service instance information according to the service request, initiates a service call, and forwards it to the service requester, specifically including:

步骤5-1,服务网关刷新服务实例缓存:服务网关定时轮询注册中心,获取最新的服务实例信息并刷新本地缓存;Step 5-1, the service gateway refreshes the service instance cache: the service gateway periodically polls the registration center to obtain the latest service instance information and refreshes the local cache;

步骤5-2,服务调用:服务请求A向服务网关发起对服务实例B的访问请求,服务网关接收到请求后,查询本地服务实例缓存,匹配最合适的服务实例B,发起服务调用,转发给服务请求A。Step 5-2, service call: Service request A initiates an access request to service instance B to the service gateway. After receiving the request, the service gateway queries the local service instance cache, matches the most suitable service instance B, initiates a service call, and forwards it to service request A.

进一步,步骤5中的服务调用,具体包括:Furthermore, the service call in step 5 specifically includes:

1)查询服务实例:服务网关根据服务请求A中的服务名、版本号信息,查询最新的服务实例信息;1) Query service instance: The service gateway queries the latest service instance information based on the service name and version number information in service request A;

2)负载均衡:根据服务所对应的负载均衡策略从服务实例列表中选择合适的服务实例B,若无可用实例则返回失败;2) Load balancing: Select the appropriate service instance B from the service instance list according to the load balancing policy corresponding to the service. If there is no available instance, it returns failure.

3)验签:服务网关对请求进行验签,验签算法采用请求路径结合时间戳、密钥的MD5签名认证,验证不通过则返回失败;3) Signature verification: The service gateway verifies the signature of the request. The signature verification algorithm uses the request path combined with the timestamp and key MD5 signature authentication. If the verification fails, it returns failure.

4)权限验证:验证API调用权限, 验证不通过则返回失败;4) Permission verification: Verify the API call permission. If the verification fails, it will return failure;

5)记录日志:服务网关记录服务调用的相关信息,包括调用时间、调用方、被调用方、请求参数;5) Logging: The service gateway records relevant information about the service call, including the call time, caller, callee, and request parameters;

6)转发请求:服务网关将服务请求A转发到服务实例B,并等待返回结果,将结果信息返回给服务请求A。6) Forward request: The service gateway forwards service request A to service instance B, waits for the return result, and returns the result information to service request A.

进一步,所述负载均衡策略包括随机、轮询和一致性哈希。Furthermore, the load balancing strategies include random, round-robin and consistent hashing.

进一步,所述注册中心采用一个协调者和多个工作者配合的分布式主从架构,协调者用于服务注册、发现和服务界面管理,协调者接受服务注册后将服务实例信息写入服务实例数据库,服务实例数据库用于存储注册中心的所有服务实例信息以及服务实例信息采集,所述服务实例数据库为闭包树表结构,用于将服务实例检索转换为动态二叉树查找算法;Furthermore, the registration center adopts a distributed master-slave architecture with a coordinator and multiple workers. The coordinator is used for service registration, discovery and service interface management. After accepting the service registration, the coordinator writes the service instance information into the service instance database. The service instance database is used to store all service instance information of the registration center and service instance information collection. The service instance database is a closure tree table structure, which is used to convert service instance retrieval into a dynamic binary tree search algorithm.

工作者节点用于与服务网关进行通讯,工作者节点通过与协调者节点定时通讯来获取全量服务实例信息并刷新注册中心本地缓存;The worker node is used to communicate with the service gateway. The worker node obtains the full service instance information and refreshes the local cache of the registration center through regular communication with the coordinator node.

与现有技术相比,本发明的有益效果在于:Compared with the prior art, the present invention has the following beneficial effects:

本发明通过搭建微服务架构,微服务架构包括服务注册器、注册中心和服务网关;服务注册器用于从不同的集群中获取服务实例信息,并同步到注册中心;所述注册中心用于提供服务管理、服务注册、服务发现、服务实例列表的服务信息,注册中心采用服务端注册方式,实现注册中心与服务的解耦;服务网关,用于从注册中心同步得到服务实例列表,选择服务实例调用;对服务本身所使用的技术栈、部署方式没有任何强制要求,并且注册中心采用服务端注册方式,由服务注册器所在集群通过事件监听或者轮询的方式获取更新后的服务实例信息,并注册到注册中心,通过扩展服务注册器来支持其他类型的集群,实现异构集群下的统一服务注册和发现,将各种不同类型和不同部署方式的服务纳入统一的微服务通讯和管理体系,通过此方式,实现了服务与注册中心的解耦,并且服务自身无需做任何改造,极大的提高了在企业复杂IT架构环境下实现系统和服务的统一管理的可操作性和便捷性,降低接入成本;The present invention builds a microservice architecture, which includes a service registrar, a registration center, and a service gateway; the service registrar is used to obtain service instance information from different clusters and synchronize it to the registration center; the registration center is used to provide service information of service management, service registration, service discovery, and service instance list, and the registration center adopts a server-side registration method to achieve decoupling of the registration center and the service; the service gateway is used to synchronize the service instance list from the registration center and select the service instance to call; there is no mandatory requirement for the technology stack and deployment method used by the service itself, and the registration center adopts a server-side registration method, and the cluster where the service registrar is located obtains the updated service instance information through event monitoring or polling, and registers it to the registration center, and supports other types of clusters by extending the service registrar to achieve unified service registration and discovery under heterogeneous clusters, and incorporates various types and services of different deployment methods into a unified microservice communication and management system. In this way, the decoupling of the service and the registration center is achieved, and the service itself does not need to be modified, which greatly improves the operability and convenience of unified management of systems and services in a complex IT architecture environment of the enterprise, and reduces the access cost;

本申请适用于异构集群部署场景下的多类型服务统一注册、发现和调用,使得各不同系统的不同服务无需进行部署和微服务架构改造,可实现统一的服务注册和发现以及服务调用。This application is applicable to the unified registration, discovery, and invocation of multiple types of services in heterogeneous cluster deployment scenarios, so that different services of different systems do not need to be deployed and transformed into microservice architecture, and unified service registration, discovery, and service invocation can be achieved.

附图说明BRIEF DESCRIPTION OF THE DRAWINGS

以下附图仅对本发明作示意性的说明和解释,并不用于限定本发明的范围,其中:The following drawings are only used to illustrate and explain the present invention, and are not used to limit the scope of the present invention, wherein:

图1:本发明不同集群的服务接入注册中心结构示意图;FIG1 is a schematic diagram of the structure of the service access registration center of different clusters of the present invention;

图2:本发明多类型服务注册和发现流程示意图;FIG2 is a schematic diagram of a multi-type service registration and discovery process of the present invention;

图3:本发明多类型服务调用流程示意图;FIG3 is a schematic diagram of a multi-type service calling process of the present invention;

图4:本发明异构集群部署场景下的多类型服务统一注册、发现和调用结构示意图。FIG4 is a schematic diagram of a unified registration, discovery and invocation structure of multiple types of services in a heterogeneous cluster deployment scenario of the present invention.

具体实施方式DETAILED DESCRIPTION

为了使本发明的目的、技术方案、设计方法及优点更加清楚明了,以下结合附图通过具体实施例对本发明进一步详细说明。应当理解,此处所描述的具体实施例仅用于解释本发明,并不用于限定本发明。In order to make the purpose, technical solution, design method and advantages of the present invention clearer, the present invention is further described in detail by specific embodiments in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

如图1-图4所示,本发明提供了一种多类型服务统一注册、发现和调用的方法,包括如下步骤:As shown in FIG. 1 to FIG. 4 , the present invention provides a method for unified registration, discovery and invocation of multiple types of services, including the following steps:

步骤1、搭建微服务架构,包括服务注册器、注册中心和服务网关;Step 1: Build a microservice architecture, including a service registrar, a registration center, and a service gateway;

所述服务注册器用于从不同的集群中获取服务实例信息,并同步到所述注册中心;所述注册中心用于提供服务管理、服务注册、服务发现以及服务实例列表的服务信息,注册中心采用服务端注册方式,由服务注册器所在集群通过事件监听或者轮询的方式获取更新后的服务实例信息,并注册到注册中心,通过扩展服务注册器来支持其他类型的集群,实现异构集群下的统一服务注册和发现,将各种不同类型和不同部署方式的服务纳入统一的微服务通讯和管理体系,通过此方式,实现注册中心与服务的解耦;所述服务网关,用于从注册中心同步得到服务实例列表,选择服务实例调用;The service registrar is used to obtain service instance information from different clusters and synchronize it to the registration center; the registration center is used to provide service information for service management, service registration, service discovery and service instance list. The registration center adopts a server-side registration method, and the cluster where the service registrar is located obtains the updated service instance information through event monitoring or polling, and registers it to the registration center. By extending the service registrar to support other types of clusters, unified service registration and discovery under heterogeneous clusters are realized, and services of various types and different deployment methods are included in a unified microservice communication and management system. In this way, the decoupling of the registration center and the service is realized; the service gateway is used to synchronize the service instance list from the registration center and select the service instance to call;

其中,所述注册中心采用一个协调者和多个工作者配合的分布式主从架构,所述协调者用于服务注册、服务发现和服务界面管理,协调者接受服务注册后将服务实例信息写入服务实例数据库,服务实例数据库用于存储注册中心的所有服务实例信息以及服务实例信息采集,所述服务实例数据库为闭包树表结构,用于将服务实例检索转换为动态二叉树查找算法;The registration center adopts a distributed master-slave architecture with a coordinator and multiple workers. The coordinator is used for service registration, service discovery and service interface management. After accepting the service registration, the coordinator writes the service instance information into the service instance database. The service instance database is used to store all service instance information of the registration center and service instance information collection. The service instance database is a closure tree table structure, which is used to convert service instance retrieval into a dynamic binary tree search algorithm.

工作者节点用于与服务网关进行通讯,用于通过工作者节点与协调者节点定时通讯来获取全量服务实例信息并刷新注册中心本地缓存;The worker node is used to communicate with the service gateway. It is used to obtain full service instance information and refresh the local cache of the registration center through regular communication between the worker node and the coordinator node.

本申请微服务架构如附图1, 图1中微服务架构包含如下:The microservice architecture of this application is shown in Figure 1. The microservice architecture in Figure 1 includes the following:

1)注册中心:注册中心提供服务管理、服务注册和服务发现的功能。此注册中心与传统的eureka(服务注册与发现)/nacos(Dynamic Naming and Configuration Service,注册配置中心)等注册中心不同,传统的注册中心多采用的客户端注册的方式,即服务自身需要与注册中心建立连接并保持心跳以实现服务的发现与注册;本申请注册中心采用服务端注册方式,即注册中心不与服务进行直接交互,由服务所在的集群,集群包括主机代理、K8s(Kubernetes,谷歌开源的容器集群管理系统)、mesos(实现这类分布式计算的框架)、阿里云acr(Alibaba Cloud Container Registry,阿里云容器镜像服务)等,通过事件监听或者轮询的方式获取服务实例的最新状态与地址,并更新到注册中心。通过此方式,实现注册中心与服务的解耦,任何类型服务均可无感接入注册中心。1) Registration center: The registration center provides service management, service registration, and service discovery functions. This registration center is different from traditional registration centers such as eureka (service registration and discovery)/nacos (Dynamic Naming and Configuration Service, registration configuration center). Traditional registration centers mostly use client registration, that is, the service itself needs to establish a connection with the registration center and maintain a heartbeat to achieve service discovery and registration; the registration center of this application adopts a server-side registration method, that is, the registration center does not interact directly with the service, and the cluster where the service is located, including the host agent, K8s (Kubernetes, Google's open source container cluster management system), mesos (a framework for implementing this type of distributed computing), Alibaba Cloud ACR (Alibaba Cloud Container Registry, Alibaba Cloud Container Mirror Service), etc., obtains the latest status and address of the service instance through event monitoring or polling, and updates it to the registration center. In this way, the registration center and the service are decoupled, and any type of service can be connected to the registration center without feeling.

2)服务注册器:为了支持图1中各种不同集群的服务接入注册中心,本申请针对不同集群提供不同的服务注册器。服务注册器主要负责从不同的集群中监听或者轮询服务的变化,并同步到注册中心。2) Service Registry: In order to support the service access registration center of various clusters in Figure 1, this application provides different service registries for different clusters. The service registrar is mainly responsible for monitoring or polling service changes from different clusters and synchronizing them to the registration center.

3)服务网关:传统的微服务调用通常是客户端负载均衡,客户端自己会存储一份服务实例清单,它是通过从注册中心进行同步得到,通过客户端内置的负载均衡算法来实现负载均衡调用。本申请中采用服务端负载均衡的方式,服务调用方与被调用方不直接通信,服务调用与被调用统一通过服务网关路由的方式实现,服务网关与注册中心定时同步服务实例清单。通过此方式,一方面实现注册中心与服务的解耦,服务调用仅需与服务网关交互; 另一方面通过服务网关,可以方便的实现服务管理、服务调用记录追踪、API(Application Programming Interface,应用程序接口)管理以及权限控制等功能;3) Service Gateway: Traditional microservice calls are usually client-side load balancing. The client itself stores a list of service instances, which is obtained by synchronization from the registration center, and the load balancing call is implemented through the client's built-in load balancing algorithm. This application adopts the server-side load balancing method. The service caller and the callee do not communicate directly. The service call and the callee are uniformly implemented through the service gateway routing. The service gateway and the registration center synchronize the service instance list regularly. In this way, on the one hand, the decoupling of the registration center and the service is achieved, and the service call only needs to interact with the service gateway; on the other hand, through the service gateway, it is easy to implement service management, service call record tracking, API (Application Programming Interface) management, and permission control functions;

需要说明的是,本申请通过搭建微服务架构,微服务架构包括注册中心、服务注册器和服务网关,注册中心采用服务端注册方式,即注册中心不与服务进行直接交互,由服务所在的集群通过事件监听或者轮询的方式获取服务实例的最新状态与地址,并更新到注册中心。通过此方式,实现注册中心与服务的解耦,任何类型服务均可无感接入注册中心;服务注册器主要负责从不同的集群中监听或者轮询的服务变化,并同步到注册中心;采用服务端负载均衡的方式,服务调用方与被调用方不直接通信,服务调用与被调用统一通过服务网关实现,服务网关与注册中心定时同步服务实例清单,其中,服务实例清单:用于记载服务配置;通过此方式,一方面实现注册中心与服务的解耦,服务调用仅需与服务网关交互; 另一方面通过服务网关,可以方便的实现服务管理、服务调用记录追踪、API管理以及权限控制等功能;通过扩展服务注册器来支持其他类型的集群,实现了异构集群中多类型服务的统一注册、发现和调用,同时提供了验签、权限验证和日志记录等安全和管理功能;It should be noted that this application builds a microservice architecture, which includes a registration center, a service registrar and a service gateway. The registration center adopts a server-side registration method, that is, the registration center does not interact directly with the service. The cluster where the service is located obtains the latest status and address of the service instance through event monitoring or polling, and updates it to the registration center. In this way, the decoupling of the registration center and the service is achieved, and any type of service can be connected to the registration center without feeling. The service registrar is mainly responsible for monitoring or polling service changes from different clusters and synchronizing them to the registration center. The server-side load balancing method is adopted, and the service caller and the callee do not communicate directly. The service call and the callee are uniformly implemented through the service gateway. The service gateway and the registration center periodically synchronize the service instance list, where the service instance list is used to record the service configuration. In this way, on the one hand, the decoupling of the registration center and the service is achieved, and the service call only needs to interact with the service gateway; on the other hand, through the service gateway, it is convenient to implement service management, service call record tracking, API management, and permission control functions; by extending the service registrar to support other types of clusters, unified registration, discovery, and calling of multiple types of services in heterogeneous clusters are achieved, and security and management functions such as signature verification, permission verification, and logging are provided.

步骤2、所述服务注册器根据服务部署所在集群,配置接入所在集群API并正确同步服务实例信息;Step 2: The service registrar configures the access cluster API and correctly synchronizes the service instance information according to the cluster where the service is deployed;

步骤3、当服务实例发生变化时,其所在集群服务实例信息与从注册中心获取全量的服务实例信息进行比对,根据比对结果更新所述服务实例信息并同步至服务注册器;Step 3: When a service instance changes, the cluster service instance information is compared with the full service instance information obtained from the registration center, and the service instance information is updated according to the comparison result and synchronized to the service registrar;

步骤4、所述服务注册器获取更新后的服务实例信息并注册到注册中心,使注册中心获得最新的服务实例信息;Step 4: The service registrar obtains the updated service instance information and registers it with the registration center, so that the registration center obtains the latest service instance information;

其中,服务注册器获取服务实例信息的模式包括:Among them, the modes in which the service registrar obtains service instance information include:

1)事件监听模式,如果集群支持服务事件监听,服务注册器会实现一个监听器并注册到集群中,以捕获服务状态变化事件,当捕获到事件时,注册器会获取相关服务实例的最新状态信息;1) Event monitoring mode: If the cluster supports service event monitoring, the service registrar will implement a listener and register it with the cluster to capture service status change events. When an event is captured, the registrar will obtain the latest status information of the relevant service instance;

2)轮询模式,如果集群不支持事件监听,服务注册器会采用轮询模式,定时调用集群的API来获取服务实例的列表和状态信息;2) Polling mode: If the cluster does not support event monitoring, the service registrar will use polling mode to periodically call the cluster API to obtain the list and status information of service instances;

在步骤4中,在将更新后的服务实例信息注册到注册中心后,还包括:In step 4, after registering the updated service instance information to the registration center, it also includes:

步骤4-1,服务注册器与注册中心同步:服务注册器将获取到的服务实例信息与注册中心中的信息进行比对,识别并处理不同状态的服务实例,当服务实例在集群中存在,但注册中心中不存在,将其添加到注册中心,识别为新增实例;当服务实例在集群中不存在,但注册中心中存在,将其从注册中心删除,识别为停止实例;当服务实例在集群和注册中心中都存在,不做处理,识别为已有实例;Step 4-1, synchronization of service registrar and registration center: the service registrar compares the obtained service instance information with the information in the registration center, identifies and processes service instances in different states. When a service instance exists in the cluster but not in the registration center, it is added to the registration center and identified as a newly added instance; when a service instance does not exist in the cluster but exists in the registration center, it is deleted from the registration center and identified as a stopped instance; when a service instance exists in both the cluster and the registration center, no processing is performed and it is identified as an existing instance;

步骤4-2,遍历和同步所有服务实例:服务注册器循环遍历所有服务实例,并重复S4-1步骤,直到所有服务实例都处理完毕;Step 4-2, traverse and synchronize all service instances: The service registrar loops through all service instances and repeats step S4-1 until all service instances are processed;

如图2所示,一个完整的多类型服务注册和发现流程如下:As shown in Figure 2, a complete multi-type service registration and discovery process is as follows:

S1,根据服务部署所在集群并启动,当发布新的服务、重启已有服务、服务扩缩容、服务由于自身故障等发生漂移时,服务所在的部署集群上的实例信息发生变化。此处依据不同的集群类型如mesos集群等均有成熟流程方案和API;S1, based on the cluster where the service is deployed and started, when new services are released, existing services are restarted, services are scaled up or down, or services drift due to their own failures, the instance information on the deployment cluster where the service is located changes. Here, there are mature process solutions and APIs based on different cluster types such as mesos clusters;

S2,当所有服务实例信息变化完成时,即旧实例停止、新服务启动完成,服务注册器获取更新后的服务实例信息并注册到注册中心;具体的:S2: When all service instance information changes are completed, that is, the old instance is stopped and the new service is started, the service registrar obtains the updated service instance information and registers it to the registration center; specifically:

当集群如mesos集群支持服务事件监听模式时,服务注册器实现监听器并注册监听集群服务状态变化事件;When a cluster such as a mesos cluster supports service event monitoring mode, the service registrar implements the listener and registers to monitor cluster service status change events;

当集群如阿里云ack(Acknowledge character,即确认字符,表示接收到的字符无错误)不支持服务事件监听时, 采用轮询模式定时调用集群API获取服务的所有实例信息。When the cluster, such as Alibaba Cloud ack (Acknowledge character, indicating that the received character is error-free), does not support service event monitoring, the cluster API is called periodically in polling mode to obtain all instance information of the service.

S3,服务注册器与注册中心同步,从注册中心获取全量的服务实例信息,并对服务进行遍历,与S2中获取的集群服务实例进行比对, 当服务名称、API版本、IP、端口等信息均一致则认为为同一实例,识别出:第一,实例在集群中存在,在注册中心不存在,此为新增实例,将此实例添加到注册中心;第二,实例在集群中不存在,在注册中心中存在,此实例已停止,将此实例从注册中心删除;第三,实例在集群和注册中心中均存在,直接跳过;S3, the service registrar synchronizes with the registration center, obtains the full amount of service instance information from the registration center, traverses the services, and compares them with the cluster service instances obtained in S2. When the service name, API version, IP, port and other information are consistent, they are considered to be the same instance. The following are identified: First, the instance exists in the cluster but not in the registration center. This is a new instance, so add this instance to the registration center; Second, the instance does not exist in the cluster but exists in the registration center. This instance has been stopped and is deleted from the registration center; Third, the instance exists in both the cluster and the registration center, so skip it directly;

S4,循环S3直到所有服务遍历结束。S4, loop S3 until all services are traversed.

步骤5、服务网关从注册中心获取最新的服务实例信息,根据服务请求匹配最合适的服务实例信息,发起服务调用,转发给服务请求方,Step 5: The service gateway obtains the latest service instance information from the registration center, matches the most appropriate service instance information according to the service request, initiates a service call, and forwards it to the service requester.

其中,服务调用方通过服务网关发起服务调用,如图3所示,一个完整的多类型服务调用流程如下:Among them, the service caller initiates a service call through the service gateway, as shown in Figure 3. A complete multi-type service call process is as follows:

步骤5-1,服务网关刷新服务实例缓存:服务网关定时轮询注册中心,获取最新的服务实例信息并刷新本地缓存;Step 5-1, the service gateway refreshes the service instance cache: the service gateway periodically polls the registration center to obtain the latest service instance information and refreshes the local cache;

步骤5-2,服务调用:服务请求A向服务网关发起对服务实例B的访问请求,服务网关接收到请求后,查询本地服务实例缓存,匹配最合适的服务实例B,发起服务调用,转发给服务请求A;Step 5-2, service call: Service request A initiates an access request to service instance B to the service gateway. After receiving the request, the service gateway queries the local service instance cache, matches the most suitable service instance B, initiates a service call, and forwards it to service request A;

步骤5-2中,服务调用,具体包括:In step 5-2, the service call specifically includes:

1)查询服务实例:服务网关根据服务请求A中的服务名和版本号信息,查询最新的服务实例信息;1) Query service instance: The service gateway queries the latest service instance information based on the service name and version number information in service request A;

2)负载均衡:根据服务所对应的负载均衡策略从服务实例列表中选择合适的服务实例B,若无可用实例,则返回失败;2) Load balancing: Select the appropriate service instance B from the service instance list according to the load balancing policy corresponding to the service. If there is no available instance, it returns failure.

其中,所述负载均衡策略包括随机、轮询和一致性哈希;The load balancing strategies include random, round-robin, and consistent hashing;

3)验签:服务网关对请求进行验签,验签算法采用请求路径结合时间戳和密钥的MD5(Message Digest Algorithm MD5,消息摘要算法第五版)签名认证,若验证不通过,则返回失败;3) Signature verification: The service gateway verifies the signature of the request. The signature verification algorithm uses the request path combined with the timestamp and the key MD5 (Message Digest Algorithm MD5, Message Digest Algorithm Version 5) signature authentication. If the verification fails, a failure is returned.

4)权限验证:验证API调用权限, 若验证不通过,则返回失败;4) Permission verification: Verify the API call permission. If the verification fails, it returns failure;

5)记录日志:服务网关记录服务调用的相关信息,包括调用时间、调用方、被调用方和请求参数;5) Logging: The service gateway records relevant information about the service call, including the call time, caller, callee, and request parameters;

6)转发请求:服务网关将服务请求A转发到服务实例B,并等待返回结果,将结果信息返回给服务请求A;6) Forwarding request: The service gateway forwards service request A to service instance B, waits for the result to be returned, and returns the result information to service request A;

在申请中,注册中心统一定义服务结构如下:In the application, the registration center uniformly defines the service structure as follows:

kind: 注册类型, 服务类型或者其他类型;kind: registration type, service type or other type;

specVersion:服务规格版本号;specVersion: service specification version number;

name: 服务名称;name: service name;

apiVersion: API版本;apiVersion: API version;

group:服务分组信息;group: service group information;

environment: 环境,如dev, test, prod等;environment: environment, such as dev, test, prod, etc.

cluster: 所属集群标识,如marathon, mesos等;cluster: the cluster ID, such as marathon, mesos, etc.

sandBox:沙箱信息;sandBox: sandbox information;

containerType: 容器类型,如docker, node等;containerType: container type, such as docker, node, etc.;

metadata: 服务元数据,如内存、cpu、环境变量、挂载点等信息;metadata: service metadata, such as memory, cpu, environment variables, mount points, etc.

loadbalance: 负载均衡策略,如随机,基于权重,基于负载等;loadbalance: load balancing strategy, such as random, weight-based, load-based, etc.

List<Instance>: 实例列表, 包含访问协议类型如Http, 容器IP,端口、负载权重等信息。List<Instance>: instance list, including access protocol type such as Http, container IP, port, load weight and other information.

如图4所示,注册中心的实现需要支持高性能和高可用,实现结构采用一admin(协调者)和多个worker(工作者)配合的分布式主从架构;As shown in Figure 4, the implementation of the registration center needs to support high performance and high availability. The implementation structure adopts a distributed master-slave architecture with one admin (coordinator) and multiple workers.

其中,协调者用于服务注册、服务界面管理等,协调者自身通过负载均衡也可以实现高可用,并且由于协调者节点只与管理端页面和服务注册器进行通讯,并发数不高,故无需做特殊性能优化。协调者在接收到服务注册后将服务实例信息写入服务实例数据库;其中,服务实例列表,用于记载服务配置数组;服务实例数据库,用于将服务实例列表存储服务实例信息及服务实例列表的采集。Among them, the coordinator is used for service registration, service interface management, etc. The coordinator itself can also achieve high availability through load balancing. And because the coordinator node only communicates with the management page and the service registrar, the number of concurrency is not high, so there is no need for special performance optimization. After receiving the service registration, the coordinator writes the service instance information into the service instance database; among them, the service instance list is used to record the service configuration array; the service instance database is used to store the service instance information and the collection of the service instance list.

服务实例数据库用于存储注册中心的所有服务以及实例信息,为了实现服务实例的高性能读写,服务实例数据库采用闭包树表结构实现,对于每个一个服务实例,生成一个key(路径), 路径key结构为:注册类型/服务规格版本号/服务名称/API版本/服务分组/环境标识/集群标识/实例ID,服务实例列表采用树表结构设计,路径中的每一级目录均作为一个独立节点,叶子节点为服务实例,服务实例数据库采用闭包树表结构将服务实例检索转换为动态二叉树查找算法;为了进一步提高查找效率, 构建Closure Table(闭包表),存储服务实例树节点的节点关系以及层级信息,通过此可以将服务实例检索时间复杂度降到O(n)级别;The service instance database is used to store all services and instance information of the registration center. In order to achieve high-performance reading and writing of service instances, the service instance database is implemented using a closure tree table structure. For each service instance, a key (path) is generated. The path key structure is: registration type/service specification version number/service name/API version/service group/environment identifier/cluster identifier/instance ID. The service instance list is designed with a tree table structure. Each level of directory in the path is an independent node, and the leaf node is the service instance. The service instance database uses a closure tree table structure to convert service instance retrieval into a dynamic binary tree search algorithm. In order to further improve the search efficiency, a Closure Table is constructed to store the node relationship and hierarchical information of the service instance tree nodes. This can reduce the service instance retrieval time complexity to the O(n) level.

一个服务实例树和闭包表数据结构如下:A service instance tree and closure table data structure is as follows:

Node //服务实例节点Node //Service instance node

{{

String parentKey; //父节点路径KeyString parentKey; //parent node path Key

String absoluteKey; //本节点路径keyString absoluteKey; //This node path key

...//实例其他信息...//Other information of the instance

List<Node>nodes; //子节点列表List<Node>nodes; //Sub-node list

}}

NodePath //服务实例树闭包表NodePath //Service instance tree closure table

{{

String ancestor; //祖先信息String ancestor; //ancestor information

String descendant;//后代信息String descendant; //descendant information

Integer depth; //树深度层级Integer depth; //Tree depth level

};};

注册中心工作者节点主要用于服务网关进行通讯,工作者节点通过与协调者节点定时通讯来获取全量服务实例信息并刷新本地缓存,工作者节点通过内存来作为服务实例缓存来提高读写效率,多个工作者节点通过外部负载均衡如nginx(轻量级/高性能的反向代理Web服务器)、SLB(Server Load Balance,网络负载均衡服务器)等来实现高可用。The worker nodes of the registration center are mainly used for communication with the service gateway. The worker nodes obtain the full service instance information and refresh the local cache through regular communication with the coordinator node. The worker nodes use memory as a service instance cache to improve read and write efficiency. Multiple worker nodes achieve high availability through external load balancing such as nginx (lightweight/high-performance reverse proxy Web server) and SLB (Server Load Balance, network load balancing server).

其中,异构集群部署场景包括但不限于:基于固定主机的容器化或者非容器化部署、基于K8s(Kubernetes,谷歌开源的容器集群管理系)或k3s集群(K8s的轻量级版本)的本地化部署、基于mesos+marathon(实现服务发现和负载平衡等功能)集群的本地化部署、基于阿里云ack/淘宝聚石塔等云原生能力的部署等;多类型服务包括但不限于基于Springcloud(全家桶式技术栈)的JAVA微服务,基于Dubbo(远程调用分布式服务框架)的JAVA微服务,使用SSH(secure shell,安全外壳协议)或者SSM(Spring、SpringMVC、Mybatis的框架整合)框架的JAVA服务,使用Node.js(Javascript代码运行环境)构建的web(通信网络)服务,基于Python(高级程序设计语言)构建的web服务等。Among them, heterogeneous cluster deployment scenarios include but are not limited to: containerized or non-containerized deployment based on fixed hosts, localized deployment based on K8s (Kubernetes, Google's open source container cluster management system) or k3s cluster (a lightweight version of K8s), localized deployment based on mesos+marathon (implementing service discovery and load balancing and other functions) clusters, deployment based on cloud native capabilities such as Alibaba Cloud ack/Taobao Jushita, etc.; multi-type services include but are not limited to JAVA microservices based on Springcloud (a full-family bucket technology stack), JAVA microservices based on Dubbo (a remote call distributed service framework), JAVA services using SSH (secure shell, secure shell protocol) or SSM (framework integration of Spring, SpringMVC, and Mybatis) frameworks, web (communication network) services built using Node.js (Javascript code running environment), web services built based on Python (a high-level programming language), etc.

值得说明的是,与现有的各种开源和商用的注册中心、微服务框架方案相比,本发明所引入的注册中心和服务网关组件,对服务本身所使用的技术栈、部署方式没有任何强制要求,并且采用服务端注册和服务网关负载的方式,实现了服务与微服务框架的解耦,服务自身并不感知微服务套件,这就给服务提供了极强的扩展性和兼容性;微服务套件自身通过高性能组件和高可用设计实现可横向扩展,通过简单方式即可实现很高的并发处理能力;It is worth noting that, compared with various existing open source and commercial registration centers and microservice framework solutions, the registration center and service gateway component introduced in the present invention has no mandatory requirements on the technology stack and deployment method used by the service itself, and adopts the server-side registration and service gateway load method to achieve the decoupling of the service and the microservice framework. The service itself does not perceive the microservice suite, which provides the service with extremely strong scalability and compatibility; the microservice suite itself can be horizontally expanded through high-performance components and high-availability design, and a high concurrent processing capability can be achieved in a simple way;

通过此方案,可以将各种不同类型和不同部署方式的服务如JAVA服务、Node(运行环境)服务等纳入统一的微服务通讯和管理体系,并且服务自身无需做任何改造,极大的提高了在企业复杂IT架构环境下实现系统和服务的统一管理的可操作性和便捷性,降低接入成本,此方案在实际业务系统集成中具有很强的创新性和实用性。Through this solution, services of various types and deployment methods, such as JAVA services, Node (operating environment) services, etc., can be incorporated into a unified microservice communication and management system, and the services themselves do not need to be modified. This greatly improves the operability and convenience of unified management of systems and services in the complex IT architecture environment of the enterprise, and reduces access costs. This solution is highly innovative and practical in actual business system integration.

实施例一: 注册中心采用JAVA(编程语言)技术和Mysql(开源的关系型数据库管理系统)数据库结构来实现,服务实例数据库采用闭包树表结构实现,实现算法为动态二叉树查找算法。Embodiment 1: The registration center is implemented using JAVA (programming language) technology and Mysql (open source relational database management system) database structure, the service instance database is implemented using a closure tree table structure, and the implementation algorithm is a dynamic binary tree search algorithm.

服务注册器,本实施例中针对K8s(Kubernetes,谷歌开源的容器集群管理系)集群说明实现的技术方法,其他集群的实现方式类似。K8s服务注册器通过引入client-javaSDK包来实现与K8s集群通讯,引入方式:Service registrar, this embodiment describes the technical method for implementing the K8s (Kubernetes, Google's open source container cluster management system) cluster, and the implementation methods of other clusters are similar. The K8s service registrar implements communication with the K8s cluster by introducing the client-javaSDK package. The introduction method is:

<dependency><dependency>

<groupId>io.kubernetes</groupId><groupId>io.kubernetes</groupId>

<artifactId>client-java</artifactId><artifactId>client-java</artifactId>

</dependency>;</dependency>;

K8s集群服务采用nodeport(端口范围)方式,来实现从集群外部不经过负载均衡或者Ingress(入口)访问到K8s集群Service(服务)内部。 注册器启动两个定时任务A和B,定时任务A定时调用注册中心API获取全量服务实例列表并缓存到本地内存;定时任务B调用K8s集群 API获取所有K8s集群服务, 对于每一个服务, 通过API获取服务发现列表,同时查询本地内存缓存获取服务实例列表,与步骤S3进行比对,从注册中心移除已停止实例和添加新增实例。The K8s cluster service uses the nodeport (port range) method to access the K8s cluster Service from outside the cluster without going through load balancing or Ingress (entrance). The registrar starts two scheduled tasks A and B. Scheduled task A regularly calls the registration center API to obtain a full list of service instances and caches them in local memory; scheduled task B calls the K8s cluster API to obtain all K8s cluster services. For each service, the service discovery list is obtained through the API, and the local memory cache is queried to obtain the service instance list, which is compared with step S3, and the stopped instances are removed from the registration center and new instances are added.

服务网关,本实施例中采用OpenResty(网络平台)脚本来实现一个高性能高可用API网关。具体的,OpenResty是一个基于 Nginx(轻量级的HTTP服务器)与Lua的高性能Web平台,其内部集成了大量精良的Lua(脚本语言)库、第三方模块以及大多数的依赖项。用于方便地搭建能够处理超高并发、扩展性极高的动态Web应用、Web 服务和动态网关。在本实施例中,可以通过编写多个lua脚本来实现对于请求的处理转发,如实现对于请求鉴权、负载均衡等,此方式均为现有技术。Service gateway, in this embodiment, OpenResty (network platform) script is used to implement a high-performance and highly available API gateway. Specifically, OpenResty is a high-performance Web platform based on Nginx (lightweight HTTP server) and Lua, which integrates a large number of sophisticated Lua (scripting language) libraries, third-party modules and most dependencies. It is used to easily build dynamic Web applications, Web services and dynamic gateways that can handle ultra-high concurrency and high scalability. In this embodiment, multiple Lua scripts can be written to realize the processing and forwarding of requests, such as request authentication, load balancing, etc. This method is all prior art.

具体实施中,该异构集群部署场景下的多类型服务统一注册、发现和调用的方法的实现从微服务架构搭建、服务注册发现、服务调用的步骤如下:In the specific implementation, the implementation of the method of unified registration, discovery and invocation of multiple types of services in the heterogeneous cluster deployment scenario is as follows from the steps of microservice architecture construction, service registration and discovery, and service invocation:

1)选择合适的技术方法实现微服务架构:注册中心、服务注册器和服务网关;1) Select appropriate technical methods to implement microservice architecture: registration center, service registrar and service gateway;

注册中心:使用Java(编程语言)技术和Mysql(开源的关系型数据库管理系统)数据库实现,采用闭包树表结构存储服务实例信息,所述注册中心用于提供服务管理、服务注册、服务发现以及服务实例列表的服务信息,注册中心采用服务端注册方式,实现注册中心与服务的解耦;Registration center: It is implemented using Java (programming language) technology and Mysql (open source relational database management system) database, and adopts a closed tree table structure to store service instance information. The registration center is used to provide service management, service registration, service discovery, and service instance list service information. The registration center adopts a server-side registration method to achieve decoupling of the registration center and the service;

服务注册器:针对K8s集群,使用client-java SDK(软件开发工具包)包与K8s集群通信。服务注册器需实现两个定时任务,定时从K8s集群获取服务实例状态信息,并同步到注册中心。Service Registry: For K8s cluster, use the client-java SDK (Software Development Kit) package to communicate with the K8s cluster. The service registrar needs to implement two scheduled tasks to obtain service instance status information from the K8s cluster at regular intervals and synchronize it to the registration center.

服务网关:采用OpenResty(网络平台)脚本实现高性能API网关,用于处理请求鉴权、负载均衡等功能,从注册中心同步得到服务实例列表,选择服务实例调用;Service Gateway: Use OpenResty (network platform) script to implement high-performance API gateway, which is used to handle request authentication, load balancing and other functions, synchronize the service instance list from the registration center, and select the service instance to call;

2)根据服务部署所在集群,配置接入服务注册器。2) Configure the access service registrar according to the cluster where the service is deployed.

根据服务部署所在的K8s集群,配置K8s服务注册器,确保服务注册器能够访问K8s集群API并正确同步服务实例信息;Configure the K8s service registrar according to the K8s cluster where the service is deployed, and ensure that the service registrar can access the K8s cluster API and correctly synchronize service instance information;

3)自动或者手动将服务发布到部署集群,服务完成启动,旧实例停止,新实例启动。3) Automatically or manually publish the service to the deployment cluster. When the service is started, the old instance is stopped and the new instance is started.

将服务自动或手动发布到K8s集群。服务发布后,旧实例停止,新实例启动;Automatically or manually publish the service to the K8s cluster. After the service is published, the old instance is stopped and the new instance is started;

4)所述服务注册器获取更新后的服务实例信息并注册到注册中心,使注册中心获得最新的服务实例信息;4) The service registrar obtains the updated service instance information and registers it with the registration center, so that the registration center obtains the latest service instance information;

具体的,K8s集群服务注册器的定时任务A从注册中心获取全量服务实例列表并缓存到本地内存。Specifically, scheduled task A of the K8s cluster service registrar obtains the full service instance list from the registration center and caches it in local memory.

定时任务B调用K8s集群API(Kubernetes Application Programming Interface,集群管理系统应用程序接口)获取所有Service(服务)和Endpoints(端点)信息。Scheduled task B calls the K8s cluster API (Kubernetes Application Programming Interface) to obtain all Service and Endpoints information.

定时任务B比对本地缓存和K8s集群中的服务实例信息,根据比对结果更新注册中心中的服务实例列表;Scheduled task B compares the service instance information in the local cache and the K8s cluster, and updates the service instance list in the registration center based on the comparison results;

5)服务网关从注册中心获取最新的服务实例信息,根据服务请求匹配最合适的服务实例信息,发起服务调用,转发给服务请求方;5) The service gateway obtains the latest service instance information from the registration center, matches the most appropriate service instance information according to the service request, initiates the service call, and forwards it to the service requester;

服务网关刷新服务实例缓存:服务网关定时轮询注册中心,获取最新的服务实例信息并刷新本地缓存;The service gateway refreshes the service instance cache: The service gateway periodically polls the registration center to obtain the latest service instance information and refreshes the local cache;

服务调用:服务请求A向服务网关发起对服务实例B的访问请求,服务网关接收到请求后,查询本地服务实例缓存,匹配最合适的服务实例B,发起服务调用,转发给服务请求A。Service call: Service request A initiates an access request to service instance B to the service gateway. After receiving the request, the service gateway queries the local service instance cache, matches the most suitable service instance B, initiates a service call, and forwards it to service request A.

需要说明的是,服务注册器,实现了服务实例状态信息与注册中心的同步,确保了服务发现的准确性;服务网关,作为服务的统一入口,实现了服务调用的路由和负载均衡,提高了系统的可扩展性和可维护性;注册中心,采用闭包树表结构存储服务实例信息,支持复杂的服务管理和查询需求。服务实例同步:通过服务注册器的定时任务实现服务实例的同步,确保注册中心中的信息与服务实际状态保持一致。多集群支持:虽然本实施例仅针对K8s集群进行了说明,但可以通过扩展服务注册器来支持其他类型的集群,实现异构集群下的统一服务注册和发现。It should be noted that the service registrar synchronizes the service instance status information with the registration center, ensuring the accuracy of service discovery; the service gateway, as a unified entry point for services, implements routing and load balancing of service calls, improving the scalability and maintainability of the system; the registration center uses a closure tree table structure to store service instance information, supporting complex service management and query requirements. Service instance synchronization: The synchronization of service instances is achieved through the scheduled tasks of the service registrar, ensuring that the information in the registration center is consistent with the actual status of the service. Multi-cluster support: Although this embodiment is only described for K8s clusters, the service registrar can be extended to support other types of clusters to achieve unified service registration and discovery under heterogeneous clusters.

值得说明的是,本申请适用于异构集群部署场景下的多类型服务统一注册、发现和调用,使得各不同系统的不同服务无需进行部署和微服务架构改造,可实现统一的服务注册和发现以及服务调用;It is worth noting that this application is applicable to the unified registration, discovery and invocation of multiple types of services in heterogeneous cluster deployment scenarios, so that different services of different systems do not need to be deployed and transformed into microservice architecture, and unified service registration and discovery as well as service invocation can be achieved;

通过搭建微服务架构,微服务架构包括服务注册器、注册中心和服务网关;服务注册器用于从不同的集群中获取服务实例信息,并同步到注册中心;所述注册中心用于提供服务管理、服务注册、服务发现、服务实例列表的服务信息,注册中心采用服务端注册方式,实现注册中心与服务的解耦;服务网关从注册中心同步得到服务实例列表,选择服务实例调用;对服务本身所使用的技术栈、部署方式没有任何强制要求,并且注册中心采用服务端注册方式,由服务所在的集群通过事件监听或者轮询的方式获取更新后的服务实例信息,并注册到注册中心,通过此方式,实现了服务与注册中心的解耦,各种不同类型和不同部署方式的服务如JAVA服务、Node.js服务等可纳入统一的注册中心,注册中心用于提供服务管理、服务注册、服务发现以及服务实例列表的服务信息,注册中心采用服务端注册方式,实现注册中心与服务的解耦;并且服务自身无需做任何改造,极大的提高了在企业复杂IT架构环境下实现系统和服务的统一管理的可操作性和便捷性,降低接入成本。By building a microservice architecture, the microservice architecture includes a service registrar, a registration center, and a service gateway; the service registrar is used to obtain service instance information from different clusters and synchronize it to the registration center; the registration center is used to provide service information for service management, service registration, service discovery, and service instance list, and the registration center adopts a server-side registration method to achieve decoupling of the registration center and the service; the service gateway synchronizes the service instance list from the registration center and selects the service instance to call; there is no mandatory requirement for the technology stack and deployment method used by the service itself, and the registration center adopts a server-side registration method, and the cluster where the service is located obtains the updated service instance information through event monitoring or polling, and registers it to the registration center. In this way, the decoupling of the service and the registration center is achieved, and services of various types and different deployment methods, such as JAVA services, Node.js services, etc. can be included in a unified registration center, and the registration center is used to provide service information for service management, service registration, service discovery, and service instance list, and the registration center adopts a server-side registration method to achieve decoupling of the registration center and the service; and the service itself does not need to be modified, which greatly improves the operability and convenience of unified management of systems and services in the complex IT architecture environment of the enterprise, and reduces the access cost.

以上已经描述了本发明的各实施例,上述说明是示例性的,并非穷尽性的,并且也不限于所披露的各实施例。在不偏离所说明的各实施例的范围和精神的情况下,对于本技术领域的普通技术人员来说许多修改和变更都是显而易见的。本文中所用术语的选择,旨在最好地解释各实施例的原理、实际应用或对市场中的技术改进,或者使本技术领域的其它普通技术人员能理解本文披露的各实施例。The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims (7)

1.一种多类型服务统一注册、发现和调用的方法,其特征在于,包括如下步骤:1. A method for unified registration, discovery and invocation of multiple types of services, characterized in that it includes the following steps: 步骤1、搭建微服务架构,包括服务注册器、注册中心和服务网关;Step 1: Build a microservice architecture, including a service registrar, a registration center, and a service gateway; 所述服务注册器用于从不同的集群中获取服务实例信息,并同步到所述注册中心;所述注册中心用于提供服务管理、服务注册、服务发现以及服务实例列表的服务信息,注册中心采用服务端注册方式,实现注册中心与服务的解耦;所述服务网关,用于从注册中心同步得到服务实例列表,选择服务实例调用;The service registrar is used to obtain service instance information from different clusters and synchronize it to the registration center; the registration center is used to provide service information for service management, service registration, service discovery and service instance list. The registration center adopts the server-side registration method to achieve the decoupling of the registration center and the service; the service gateway is used to synchronize the service instance list from the registration center and select the service instance to call; 步骤2、所述服务注册器根据服务部署所在集群,配置接入所在集群API并正确同步服务实例信息;Step 2: The service registrar configures the access cluster API and correctly synchronizes the service instance information according to the cluster where the service is deployed; 步骤3、当服务实例发生变化时,其所在集群服务实例信息与从注册中心获取全量的服务实例信息进行比对,根据比对结果更新所述服务实例信息并同步至服务注册器;Step 3: When a service instance changes, the cluster service instance information is compared with the full service instance information obtained from the registration center, and the service instance information is updated according to the comparison result and synchronized to the service registrar; 步骤4、所述服务注册器获取更新后的服务实例信息并注册到注册中心,使注册中心获得最新的服务实例信息;Step 4: The service registrar obtains the updated service instance information and registers it with the registration center, so that the registration center obtains the latest service instance information; 步骤5、服务网关从注册中心获取最新的服务实例信息,根据服务请求匹配最合适的服务实例信息,发起服务调用,转发给服务请求方。Step 5: The service gateway obtains the latest service instance information from the registration center, matches the most appropriate service instance information according to the service request, initiates a service call, and forwards it to the service requester. 2.根据权利要求1所述的多类型服务统一注册、发现和调用的方法,其特征在于,2. The method for unified registration, discovery and invocation of multiple types of services according to claim 1, characterized in that: 在步骤4中,服务注册器获取服务实例信息的模式包括:In step 4, the mode in which the service registrar obtains service instance information includes: 1)事件监听模式,如果集群支持服务事件监听,服务注册器会实现一个监听器并注册到集群中,以捕获服务状态变化事件,当捕获到事件时,注册器会获取相关服务实例的最新状态信息;1) Event monitoring mode: If the cluster supports service event monitoring, the service registrar will implement a listener and register it with the cluster to capture service status change events. When an event is captured, the registrar will obtain the latest status information of the relevant service instance; 2)轮询模式,如果集群不支持事件监听,服务注册器会采用轮询模式,定时调用集群的API来获取服务实例的列表和状态信息。2) Polling mode: If the cluster does not support event monitoring, the service registrar will use polling mode to periodically call the cluster API to obtain the list and status information of service instances. 3.根据权利要求2所述的一种多类型服务统一注册、发现和调用的方法,其特征在于,在步骤4中,在将更新后的服务实例信息注册到注册中心后,还包括:3. According to claim 2, a method for unified registration, discovery and invocation of multiple types of services is characterized in that in step 4, after registering the updated service instance information to the registration center, it also includes: 步骤4-1,服务注册器与注册中心同步:服务注册器将获取到的服务实例信息与注册中心中的信息进行比对,识别并处理不同状态的服务实例,当服务实例在集群中存在,但注册中心中不存在,将其添加到注册中心,识别为新增实例;当服务实例在集群中不存在,但注册中心中存在,将其从注册中心删除,识别为停止实例;当服务实例在集群和注册中心中都存在,不做处理,识别为已有实例;Step 4-1, synchronization of service registrar and registration center: the service registrar compares the obtained service instance information with the information in the registration center, identifies and processes service instances in different states. When a service instance exists in the cluster but not in the registration center, it is added to the registration center and identified as a newly added instance; when a service instance does not exist in the cluster but exists in the registration center, it is deleted from the registration center and identified as a stopped instance; when a service instance exists in both the cluster and the registration center, no processing is performed and it is identified as an existing instance; 步骤4-2,遍历和同步所有服务实例:服务注册器循环遍历所有服务实例,并重复S4-1步骤,直到所有服务实例都处理完毕。Step 4-2, traverse and synchronize all service instances: The service registrar loops through all service instances and repeats step S4-1 until all service instances are processed. 4.根据权利要求3所述的多类型服务统一注册、发现和调用的方法,其特征在于,4. The method for unified registration, discovery and invocation of multiple types of services according to claim 3, characterized in that: 在步骤5中,服务网关从注册中心获取最新的服务实例信息,根据服务请求匹配最合适的服务实例信息,发起服务调用,转发给服务请求方,具体包括:In step 5, the service gateway obtains the latest service instance information from the registration center, matches the most appropriate service instance information according to the service request, initiates a service call, and forwards it to the service requester, including: 步骤5-1,服务网关刷新服务实例缓存:服务网关定时轮询注册中心,获取最新的服务实例信息并刷新本地缓存;Step 5-1, the service gateway refreshes the service instance cache: the service gateway periodically polls the registration center to obtain the latest service instance information and refreshes the local cache; 步骤5-2,服务调用:服务请求A向服务网关发起对服务实例B的访问请求,服务网关接收到请求后,查询本地服务实例缓存,匹配最合适的服务实例B,发起服务调用,转发给服务请求A。Step 5-2, service call: Service request A initiates an access request to service instance B to the service gateway. After receiving the request, the service gateway queries the local service instance cache, matches the most suitable service instance B, initiates a service call, and forwards it to service request A. 5.根据权利要求4所述的多类型服务统一注册、发现和调用的方法,其特征在于,5. The method for unified registration, discovery and invocation of multiple types of services according to claim 4, characterized in that: 步骤5中的服务调用,具体包括:The service call in step 5 specifically includes: 1)查询服务实例:服务网关根据服务请求A中的服务名和版本号信息,查询最新的服务实例信息;1) Query service instance: The service gateway queries the latest service instance information based on the service name and version number information in service request A; 2)负载均衡:根据服务所对应的负载均衡策略从服务实例列表中选择合适的服务实例B,若无可用实例,则返回失败;2) Load balancing: Select the appropriate service instance B from the service instance list according to the load balancing policy corresponding to the service. If there is no available instance, it returns failure. 3)验签:服务网关对请求进行验签,验签算法采用请求路径结合时间戳和密钥的MD5签名认证,若验证不通过,则返回失败;3) Signature verification: The service gateway verifies the signature of the request. The signature verification algorithm uses the request path combined with the timestamp and the MD5 signature authentication of the key. If the verification fails, a failure is returned. 4)权限验证:验证API调用权限, 若验证不通过,则返回失败;4) Permission verification: Verify the API call permission. If the verification fails, it returns failure; 5)记录日志:服务网关记录服务调用的相关信息,包括调用时间、调用方、被调用方和请求参数;5) Logging: The service gateway records relevant information about the service call, including the call time, caller, callee, and request parameters; 6)转发请求:服务网关将服务请求A转发到服务实例B,并等待返回结果,将结果信息返回给服务请求A。6) Forward request: The service gateway forwards service request A to service instance B, waits for the return result, and returns the result information to service request A. 6.根据权利要求5所述的多类型服务统一注册、发现和调用的方法,其特征在于,6. The method for unified registration, discovery and invocation of multiple types of services according to claim 5, characterized in that: 所述负载均衡策略包括随机、轮询和一致性哈希。The load balancing strategies include random, round-robin, and consistent hashing. 7.根据权利要求1所述的多类型服务统一注册、发现和调用的方法,其特征在于,7. The method for unified registration, discovery and invocation of multiple types of services according to claim 1, characterized in that: 所述注册中心采用一个协调者和多个工作者配合的分布式主从架构,所述协调者用于服务注册、服务发现和服务界面管理,协调者接受服务注册后将服务实例信息写入服务实例数据库,服务实例数据库用于存储注册中心的所有服务实例信息以及服务实例信息采集,所述服务实例数据库为闭包树表结构,用于将服务实例检索转换为动态二叉树查找算法;The registration center adopts a distributed master-slave architecture with a coordinator and multiple workers. The coordinator is used for service registration, service discovery and service interface management. After accepting the service registration, the coordinator writes the service instance information into the service instance database. The service instance database is used to store all service instance information of the registration center and service instance information collection. The service instance database is a closure tree table structure, which is used to convert service instance retrieval into a dynamic binary tree search algorithm; 工作者节点用于与服务网关进行通讯,工作者节点通过与协调者节点定时通讯来获取全量服务实例信息并刷新注册中心本地缓存。The worker node is used to communicate with the service gateway. The worker node obtains the full service instance information and refreshes the local cache of the registration center by regularly communicating with the coordinator node.
CN202411189600.7A 2024-08-28 2024-08-28 Method for unified registration, discovery and calling of multi-type service Active CN118714191B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202411189600.7A CN118714191B (en) 2024-08-28 2024-08-28 Method for unified registration, discovery and calling of multi-type service

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202411189600.7A CN118714191B (en) 2024-08-28 2024-08-28 Method for unified registration, discovery and calling of multi-type service

Publications (2)

Publication Number Publication Date
CN118714191A true CN118714191A (en) 2024-09-27
CN118714191B CN118714191B (en) 2024-11-15

Family

ID=92822425

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202411189600.7A Active CN118714191B (en) 2024-08-28 2024-08-28 Method for unified registration, discovery and calling of multi-type service

Country Status (1)

Country Link
CN (1) CN118714191B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN119292696A (en) * 2024-10-09 2025-01-10 北京计算机技术及应用研究所 A method for quickly building a lightweight service catalog based on the Nacos framework
CN119363813A (en) * 2024-10-24 2025-01-24 中国民航信息网络股份有限公司 Service discovery method and device based on cache mechanism, and electronic device

Citations (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107102847A (en) * 2016-02-23 2017-08-29 中国水电工程顾问集团有限公司 Software development methodology, apparatus and system based on micro services
CN107911430A (en) * 2017-11-06 2018-04-13 上海电机学院 A kind of micro services infrastructure equipment
CN108390766A (en) * 2017-10-25 2018-08-10 国云科技股份有限公司 Service registration and discovery method suitable for micro-service
CN109788055A (en) * 2019-01-11 2019-05-21 武汉虹旭信息技术有限责任公司 A kind of service governing system and its method based on micro services framework
CN111176873A (en) * 2019-12-20 2020-05-19 中国平安财产保险股份有限公司 Micro-service automatic offline method and device, computer equipment and storage medium
CN111190736A (en) * 2019-12-31 2020-05-22 苏宁云计算有限公司 Low-intrusion distributed timing task scheduling system and method based on microservice
WO2020147331A1 (en) * 2019-01-18 2020-07-23 苏宁云计算有限公司 Micro-service monitoring method and system
CN111800462A (en) * 2020-05-28 2020-10-20 中国平安财产保险股份有限公司 Micro-service instance processing method and device, computer equipment and storage medium
CN112087333A (en) * 2020-09-07 2020-12-15 上海浦东发展银行股份有限公司 Micro-service registration center cluster and information processing method thereof
WO2021159657A1 (en) * 2020-02-14 2021-08-19 平安科技(深圳)有限公司 Method and device for microservice deployment, computer device, and storage medium
WO2021179493A1 (en) * 2020-03-09 2021-09-16 平安科技(深圳)有限公司 Microservice-based load balancing method, apparatus and device, and storage medium
WO2022134358A1 (en) * 2020-12-24 2022-06-30 平安科技(深圳)有限公司 Microservice data processing method, apparatus, microservice processing platform, and medium
CN114816735A (en) * 2022-03-30 2022-07-29 南京南审审计大数据研究院有限公司 System and method for executing data analysis task based on Nacos distributed cluster
CN115103008A (en) * 2022-06-13 2022-09-23 北京奇艺世纪科技有限公司 Service request forwarding system
CN115643306A (en) * 2022-09-07 2023-01-24 上海浦东发展银行股份有限公司 Dynamic issuing method for grid service instance and configuration
CN116155978A (en) * 2023-02-13 2023-05-23 京东科技信息技术有限公司 Multi-registry adaptation method, device, electronic equipment and storage medium
CN116743844A (en) * 2023-08-01 2023-09-12 中国光大银行股份有限公司 Service calling device and method of distributed system
CN117278640A (en) * 2023-09-05 2023-12-22 北京长河数智科技有限责任公司 An API interface calling method and system based on data collection
WO2023246168A1 (en) * 2022-06-23 2023-12-28 华为云计算技术有限公司 Micro-service migration method and apparatus, and computing device
WO2024068023A1 (en) * 2022-09-30 2024-04-04 Lenovo (Singapore) Pte. Ltd. Inter-plane service based architecture in a wireless communications network

Patent Citations (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107102847A (en) * 2016-02-23 2017-08-29 中国水电工程顾问集团有限公司 Software development methodology, apparatus and system based on micro services
CN108390766A (en) * 2017-10-25 2018-08-10 国云科技股份有限公司 Service registration and discovery method suitable for micro-service
CN107911430A (en) * 2017-11-06 2018-04-13 上海电机学院 A kind of micro services infrastructure equipment
CN109788055A (en) * 2019-01-11 2019-05-21 武汉虹旭信息技术有限责任公司 A kind of service governing system and its method based on micro services framework
WO2020147331A1 (en) * 2019-01-18 2020-07-23 苏宁云计算有限公司 Micro-service monitoring method and system
CN111176873A (en) * 2019-12-20 2020-05-19 中国平安财产保险股份有限公司 Micro-service automatic offline method and device, computer equipment and storage medium
CN111190736A (en) * 2019-12-31 2020-05-22 苏宁云计算有限公司 Low-intrusion distributed timing task scheduling system and method based on microservice
WO2021159657A1 (en) * 2020-02-14 2021-08-19 平安科技(深圳)有限公司 Method and device for microservice deployment, computer device, and storage medium
WO2021179493A1 (en) * 2020-03-09 2021-09-16 平安科技(深圳)有限公司 Microservice-based load balancing method, apparatus and device, and storage medium
CN111800462A (en) * 2020-05-28 2020-10-20 中国平安财产保险股份有限公司 Micro-service instance processing method and device, computer equipment and storage medium
CN112087333A (en) * 2020-09-07 2020-12-15 上海浦东发展银行股份有限公司 Micro-service registration center cluster and information processing method thereof
WO2022134358A1 (en) * 2020-12-24 2022-06-30 平安科技(深圳)有限公司 Microservice data processing method, apparatus, microservice processing platform, and medium
CN114816735A (en) * 2022-03-30 2022-07-29 南京南审审计大数据研究院有限公司 System and method for executing data analysis task based on Nacos distributed cluster
CN115103008A (en) * 2022-06-13 2022-09-23 北京奇艺世纪科技有限公司 Service request forwarding system
WO2023246168A1 (en) * 2022-06-23 2023-12-28 华为云计算技术有限公司 Micro-service migration method and apparatus, and computing device
CN115643306A (en) * 2022-09-07 2023-01-24 上海浦东发展银行股份有限公司 Dynamic issuing method for grid service instance and configuration
WO2024068023A1 (en) * 2022-09-30 2024-04-04 Lenovo (Singapore) Pte. Ltd. Inter-plane service based architecture in a wireless communications network
CN116155978A (en) * 2023-02-13 2023-05-23 京东科技信息技术有限公司 Multi-registry adaptation method, device, electronic equipment and storage medium
CN116743844A (en) * 2023-08-01 2023-09-12 中国光大银行股份有限公司 Service calling device and method of distributed system
CN117278640A (en) * 2023-09-05 2023-12-22 北京长河数智科技有限责任公司 An API interface calling method and system based on data collection

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN119292696A (en) * 2024-10-09 2025-01-10 北京计算机技术及应用研究所 A method for quickly building a lightweight service catalog based on the Nacos framework
CN119363813A (en) * 2024-10-24 2025-01-24 中国民航信息网络股份有限公司 Service discovery method and device based on cache mechanism, and electronic device

Also Published As

Publication number Publication date
CN118714191B (en) 2024-11-15

Similar Documents

Publication Publication Date Title
CN109981716B (en) A microservice calling method and device
US11086531B2 (en) Scaling events for hosting hierarchical data structures
US8555242B2 (en) Decentralized system services
JP5624479B2 (en) Sync server process
CN118714191A (en) A method for unified registration, discovery and invocation of multiple types of services
CN114401098B (en) Application system and method for quickly constructing microservice
US8977686B2 (en) Distributed routing table interface
EP0926608B1 (en) Distributed persistent storage for intermittently connected clients
US9749445B2 (en) System and method for updating service information for across-domain messaging in a transactional middleware machine environment
CN112149079A (en) Planning review management platform and user access authorization method based on microservice architecture
CN106776720A (en) A kind of document handling method and device
US20060259523A1 (en) System and method of synchronization of internal data cache with wireless device application data repositories
CN116204239A (en) Service processing method, device and computer readable storage medium
CN110737510B (en) block device management system
CN107045466A (en) Auditing method, the apparatus and system of business datum
WO2021093671A1 (en) Task processing method, system, apparatus and device, and computer readable storage medium
US11663058B1 (en) Preemptive filtering of events of an event bus with a deterministic filter
CN118170557B (en) Nacos-based distributed lock implementation method, distributed lock and system
CN116069753A (en) Storage-computing separation method, system, equipment and medium
CN116185586A (en) Resource object scheduling method, system, cluster and computer readable medium
CN116866415A (en) Service management method and system
CN117318964A (en) Method, device, electronic equipment and medium for resource access and data processing
Dittrich A Distributed Low-Level Data Structure for Composing Scalable Concurrent Web Services
HK1184611B (en) Mock service system and method for processing mock service
HK1184611A (en) Mock service system and method for processing mock service

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant