emo

emo

HomeArchives

Skynet Source Code Analysis - Structural Analysis

296
AI Translation
This post is translated from Chinese into English through AI.View Original
AI-generated summary
The text provides an overview of the Skynet framework, including its source code, framework structure, core modules, service management, Lua extension modules, and network communication. It discusses the registration and lookup of services, message processing, and various Lua extension modules. It also briefly mentions the network model used by Skynet. The text discusses the Actor model and its related concepts, including message queues and the skynet_context structure. It explains the message passing process, the logging module, and the memory management system in Skynet. It mentions Skynet's use of high-performance analysis techniques, I/O multiplexing models, non-blocking I/O models, and message queues for communication between actors.

Source code address: https://github.com/cloudwu/skynet

Framework Structure#

  • skynet-src: The core source code of Skynet, including the main framework, service management, network communication, timer, and other modules.
  • 3rd: Third-party libraries that Skynet depends on, including LuaJIT, luasocket, etc.
  • config: Skynet's configuration file directory.
  • example: Skynet's example code directory, including echo service, pingpong example, socket module example, etc.
  • luaclib: Skynet's C extension libraries, such as cjson, protobuf, etc.
  • lualib: Skynet's Lua extension libraries, such as skynet, socket, etc.
  • service: Skynet's service code directory, including Lua source code and initialization scripts for each service.
  • test: Skynet's test code directory.

Core Modules#

  • skynet_main.c: The main function of Skynet, including parameter parsing, initialization, and service startup.
  • skynet.h: The core header file of Skynet, including data structure definitions, function declarations, etc.
  • service-src: Skynet service management module, including service registration, service lookup, message processing, etc.
  • lualib-src: Skynet Lua extension module, including functions for interaction between Lua and C.
  • timer.c: Skynet timer module, including timer setting, deletion, etc.
  • socket_server.c: Skynet network communication module, including TCP/UDP network connection, message processing, etc.
  • logger.c: Skynet logging module, including logging recording, output, etc.
  • malloc_hook.c: Skynet memory management hook, used to track memory usage.

Service Management#

  • Register Service: Register a service in the framework so that other services can call it.
  • Find Service: Find the corresponding service handle based on the service name.
  • Message Processing: Handle messages from other services.

Lua Extension Modules#

  • skynet.lua: The core interface of Skynet, including service registration, message sending, timers, etc.
  • socket.lua: Network communication module, including support for TCP, UDP, HTTP, etc.
  • cjson.lua: JSON parsing library.
  • protobuf.lua: Lua implementation of Google's Protobuf library.

Network Communication#

1. Network Model

  • skynet_socket: Structure of a socket, including socket ID, type, address, etc.
  • struct skynet_context: Context structure of an actor, including actor name, message queue, etc.
  • skynet_socket_poll: Socket polling function for handling network events.
  • skynet_socket_send: Send data to the specified socket.
  • skynet_socket_connect: Connect the socket to the specified address and port number.
  • skynet_socket_bind: Bind the socket to the specified address and port number.
  • skynet_socket_listen: Start listening on the socket.
  • skynet_socket_close: Close the specified socket.

2. Network Communication

  • skynet_send: Send a message to the specified actor. This function will find the target actor and add the message to its message queue.
  • skynet_socket_start: Start the network service and listen on the specified port number.
  • skynet_socket_poller: Network polling function for handling network events.
  • skynet_socket_udp: Create a UDP socket and bind it to the specified address and port number.
  • skynet_socket_udp_connect: Connect the UDP socket to the specified address and port number.
  • skynet_socket_udp_send: Send data to the specified UDP socket.
  • skynet_socket_udp_address: Get the address information of the UDP socket.
  • skynet_socket_udp_close: Close the specified UDP socket.

Actor Model Related#

1. Message Queue

  • struct message: Structure of a message, including source, destination, type, data, etc.

    The message queue stores message structures, which include information such as the source, destination, type, and data of the message, facilitating message passing and processing.

  • struct message_queue: Structure of a message queue, including the head and tail pointers, message count, etc.

    The message_queue structure is the management structure of the message queue, including the head and tail pointers, message count, etc. It is used to manage the messages in the message queue, facilitating operations such as adding and popping messages.

  • skynet_mq_push: Add a message to the message queue.

    The skynet_mq_push function is used to add a message to the message queue, adding the message to the tail of the message queue for easy processing.

  • skynet_mq_pop: Pop a message from the message queue.

    The skynet_mq_pop function is used to pop a message from the message queue, removing the message from the head of the message queue for processing.

  • skynet_mq_length: Get the number of messages in the message queue.

    The skynet_mq_length function is used to get the number of messages in the message queue, facilitating the counting of messages.

  • skynet_mq_alloc: Allocate a message queue from the memory pool.

    The skynet_mq_alloc function is used to allocate a new message queue from the memory pool for storing and managing messages.

  • skynet_mq_mark_release: Mark the message queue as release status. When all messages in the message queue have been processed, it can be released back to the memory pool.

    The skynet_mq_mark_release function is used to mark the message queue as release status. When all messages in the message queue have been processed, it can be released back to the memory pool for memory management.

  • skynet_mq_release: Release message queues that are marked as release status.

    The skynet_mq_release function is used to release message queues that are marked as release status, facilitating memory management.

2. Actor Model Communication Mechanism

  • struct skynet_context: Context structure of an actor, including actor name, message queue, etc.

    The skynet_context structure is the context structure of an actor, including actor name, message queue, etc. It is used to manage the state and information of the actor.

  • skynet_context_new: Create a new actor and initialize its context structure.

    The skynet_context_new function is used to create a new actor and initialize its context structure, facilitating the management and use of actors.

  • skynet_context_release: Release the context structure of an actor.

    The skynet_context_release function is used to release the context structure of an actor, facilitating memory management.

  • skynet_context_push: Add a message to the message queue of an actor.

    The skynet_context_push function is used to add a message to the message queue of an actor, facilitating message passing and processing.

  • skynet_context_pop: Pop a message from the message queue of an actor.

    The skynet_context_pop function is used to pop a message from the message queue of an actor, facilitating message processing.

  • skynet_context_handle_message: Handle all messages in the message queue of an actor.

    The skynet_context_handle_message function is used to handle all messages in the message queue of an actor, facilitating unified message processing.

  • skynet_send: Send a message to the specified actor. This function will find the target actor and add the message to its message queue.

    The skynet_send function is used to send a message to the specified actor. This function will find the target actor and add the message to its message queue, facilitating message passing and processing.

3. Message Passing Process

  1. The sender uses skynet_send to send a message to the specified actor.
  2. Skynet finds the context structure of the target actor based on the destination of the message and adds the message to its message queue.
  3. The receiver uses skynet_context_pop to pop a message from its own message queue and process it.

Logging Module#

  • Records the running status and error information of various services.

Memory Management#

Skynet's memory management module uses a transparent memory pool implementation, which can effectively reduce the frequency of memory allocation and deallocation, and improve memory usage efficiency. The analysis is as follows:

  1. Structure of the Memory Pool
  • struct mem_data: Structure of a block in the memory pool, including previous and next links, size, etc.
  • struct mem_env: Management structure of the memory pool, including the starting address, ending address, block size, etc.
  1. Creation of the Memory Pool
  • mem_init: Initialize the management structure of the memory pool, set the starting address, ending address, block size, etc.
  • mem_alloc: Allocate a certain size of memory from the memory pool and return a pointer to an available block in the memory pool.
  • mem_free: Free a block of memory back to the memory pool and mark it as a free block.
  1. Expansion of the Memory Pool
  • mem_newpool: Allocate a new memory pool after the management structure of the memory pool and add it to the memory pool chain, to support the expansion of the memory pool.
  1. Usage of the Memory Pool
  • skynet_malloc: Allocate a block of memory of the specified size using the memory pool.
  • skynet_calloc: Allocate a block of memory of the specified size using the memory pool and clear it.
  • skynet_free: Free a specified block of memory back to the memory pool.
  1. Memory Pool Statistics
  • mem_data_dump: Output the usage information of each block in the memory pool.
  • mem_report: Output the overall information of the memory pool, including usage rate, total size, number of free blocks, etc.
  1. Principle of the Memory Pool

The memory pool implementation principle is to divide a continuous block of memory into blocks of equal size, and use previous and next links to connect these blocks. Free blocks are connected by a linked list, while allocated blocks are not in the linked list. The management structure of the memory pool includes the starting address, ending address, block size, etc. When allocating and freeing memory from the memory pool, only the pointers of the linked list need to be modified, without the need for memory allocation and deallocation operations, achieving fast memory allocation and deallocation effects. At the same time, the use of memory pools can reduce memory fragmentation and improve memory usage efficiency.

High Concurrency, High Throughput, High Performance Analysis#

  • skynet_socket_poller uses I/O multiplexing model and supports epoll, kqueue, and select modes, which can effectively improve the concurrent processing capacity of network communication.
  • Skynet uses non-blocking I/O model and asynchronous I/O functions provided by the underlying operating system, avoiding the blocking of threads during I/O operations, and improving system throughput and performance.
  • Skynet uses message queues as the data structure for communication between actors, avoiding thread competition and the use of locks, further improving system concurrency and performance.
Loading...
Ownership of this post data is guaranteed by blockchain and smart contracts to the creator alone.