{{tag>dragengine networking}} [[:start|Start Page]] >> [[:gamedev|Game Development with the Drag[en]gine]] >> **Drag[en]gine Network Protocol Specification** ====== Drag[en]gine Network Protocol Specification ====== The Drag[en]gine Network Protocol (DNP) provides communication support for games using an unreliable communication channel, in general UDP. This includes synchronizing game states (//Network States// containing //Network State Values//) and sending messages. Synchronizing network state values is handled internally by the communication modules involved. Sending messages the application can choose if this is done reliable or unreliable. Network protocols typically use //Network Byte Order// as defined by the TCP standard which is //Big Endian//. The message writing and reading of DNP is implemented to match Drag[en]gine #@LinkApiDocDE2_HTML~classdecBaseFileReader.html,File Reader/Writer~@# system. This allows all data written by game scripts to be stored to file, written to memory and transmitted across the network without changing the data. Since this system uses //Little Endian// DNP uses //Little Endian// too. Hence throughout this specification //Little Endian// byte ordering is used unless marked otherwise. System intending to communicate with Drag[en]gine games can use [[https://github.com/LordOfDragons/denetwork|Drag[en]gine Networking Library]] to use DNP without implementing the specification outlined here. You need this specification only if you want to implement DNP into your application without this library. ====== Data Types ====== For message writing and reading the same #@LinkApiDocDE2_HTML~classdecBaseFileWriter.html,file writer~@# system is used as for file and memory data writing and reading. This ensures any data written in the game engine (no matter if saved to files, persisted to memory or transmitted across the network) is using the same binary format and is interchangeable. You can save data to files and transmit the file data as-is across the network and read it successfully on the other side. The following data types are supported (names match #@LinkApiDocDE2_HTML~classdecBaseFileReader.html,decBaseFileReader~@#: ^ Data type ^ Length ^ Description ^ | Char | 1 byte | Signed 8-bit integer in the range from -128 to 127 inclusive. | | Byte | 1 byte | Unsigned 8-bit integer in the range from 0 to 255 inclusive. | | Short | 2 bytes | Signed 16-bit integer in the range from -32'768 to 32'767 inclusive. | | UShort | 2 bytes | Unsigned 16-bit integer in the range from 0 to 65'535 inclusive. | | Int | 4 bytes | Signed 32-bit integer in the range from -2'147'483'648 to 2'147'483'647 inclusive. | | UInt | 4 bytes | Unsigned 32-bit integer in the range from 0 to 4'294'967'295 inclusive. | | Long | 8 bytes | Signed 64-bit integer in the range from -pow(2,63) to pow(2,63)-1 inclusive. | | ULong | 8 bytes | Unsigned 64-bit integer in the range from 0 to 18'446'744'073'709'551'615 inclusive. | | Float | 4 bytes | 32-bit floating point value. | | Double | 8 bytes | 64-bit floating point value. | | String8 | 1 + length bytes | Unsigned 8-bit value storing the length of UTF-8 encoded string followed by "length" unsigned 8-bit integer characters. #@LinkApiDocDE2_HTML~classdecString.html,String~@# can not be longer than 255 characters (UTF8 encoded). | | String16 | 2 + length bytes | Unsigned 16-bit value storing the length of UTF-8 encoded string followed by "length" unsigned 8-bit integer characters. #@LinkApiDocDE2_HTML~classdecString.html,String~@# can not be longer than 65'535 characters (UTF8 encoded). | | Vector | 12 bytes | 3-Component #@LinkApiDocDE2_HTML~classdecVector.html,vector~@# with 32-bit floating point values for each component stored in order X, Y then Z. | | Vector2 | 8 bytes | 2-Component #@LinkApiDocDE2_HTML~classdecVector2.html,vector~@# with 32-bit floating point values for each component stored in order X then Y. | | Quaternion | 16 bytes | 4-Component #@LinkApiDocDE2_HTML~classdecQuaternion.html,quaternion~@# with 32-bit floating point values for each component stored in order X, Y, Z then W. | | Point | 8 bytes | 2-Component #@LinkApiDocDE2_HTML~classdecPoint.html,point~@# with 32-bit signed integer values for each component stored in order X then Y. | | Point3 | 12 bytes | 3-Component #@LinkApiDocDE2_HTML~classdecPoint3.html,point~@# with 32-bit signed integer values for each component stored in order X, Y then Z. | | DVector | 24 bytes | 3-Component #@LinkApiDocDE2_HTML~classdecDVector.html,vector~@# with 64-bit floating point values for each component stored in order X, Y then Z. | | Color | 16 bytes | 4-Component #@LinkApiDocDE2_HTML~classdecColor.html,color~@# with 32-bit floating point values for each component stored in order R, G, B then A. | | Color3 | 12 bytes | 3-Component #@LinkApiDocDE2_HTML~classdecColor.html,color~@# with 32-bit floating point values for each component stored in order R, G then B. | ====== State Value Data Types ====== State values can use more diverse data types to allow network modules to better optimize network usage. These data types are supported for state values: ^ Data type ^ Code ^ Length ^ Description ^ | SInt8 | ''0'' | 1 byte | Signed 8-bit integer in the range from -128 to 127 inclusive. | | UInt8 | ''1'' | 1 byte | Unsigned 8-bit integer in the range from 0 to 255 inclusive. | | SInt16 | ''2'' | 2 bytes | Signed 16-bit integer in the range from -32'768 to 32'767 inclusive. | | UInt16 | ''3'' | 2 bytes | Unsigned 16-bit integer in the range from 0 to 65'535 inclusive. | | SInt32 | ''4'' | 4 bytes | Signed 32-bit integer in the range from -2'147'483'648 to 2'147'483'647 inclusive. | | UInt32 | ''5'' | 4 bytes | Unsigned 32-bit integer in the range from 0 to 4'294'967'295 inclusive. | | SInt64 | ''6'' | 8 bytes | Signed 64-bit integer in the range from -pow(2,63) to pow(2,63)-1 inclusive. | | UInt64 | ''7'' | 8 bytes | Unsigned 64-bit integer in the range from 0 to 18'446'744'073'709'551'615 inclusive. | | Float16 | ''8'' | 2 bytes | 16-bit floating point value. | | Float32 | ''9'' | 4 bytes | 32-bit floating point value. | | Float64 | ''10'' | 8 bytes | 64-bit floating point value. | | String | ''11'' | N bytes | UTF8 encoded #@LinkApiDocDE2_HTML~classdecString.html,String~@# of variable length. | | Data | ''12'' | N bytes | Arbitrary data stored as UInt8 values. | | Point2S8 | ''13'' | 2 bytes | 2-Component #@LinkApiDocDE2_HTML~classdecPoint.html,point~@# with signed 8-bit signed integer values for each component stored in order X then Y. | | Point2U8 | ''14'' | 2 bytes | 2-Component #@LinkApiDocDE2_HTML~classdecPoint.html,point~@# with unsigned 8-bit signed integer values for each component stored in order X then Y. | | Point2S16 | ''15'' | 4 bytes | 2-Component #@LinkApiDocDE2_HTML~classdecPoint.html,point~@# with signed 16-bit signed integer values for each component stored in order X then Y. | | Point2U16 | ''16'' | 4 bytes | 2-Component #@LinkApiDocDE2_HTML~classdecPoint.html,point~@# with unsigned 16-bit signed integer values for each component stored in order X then Y. | | Point2S32 | ''17'' | 8 bytes | 2-Component #@LinkApiDocDE2_HTML~classdecPoint.html,point~@# with signed 32-bit signed integer values for each component stored in order X then Y. | | Point2U32 | ''18'' | 8 bytes | 2-Component #@LinkApiDocDE2_HTML~classdecPoint.html,point~@# with unsigned 32-bit signed integer values for each component stored in order X then Y. | | Point2S64 | ''19'' | 16 bytes | 2-Component #@LinkApiDocDE2_HTML~classdecPoint.html,point~@# with signed 64-bit signed integer values for each component stored in order X then Y. | | Point2U64 | ''20'' | 16 bytes | 2-Component #@LinkApiDocDE2_HTML~classdecPoint.html,point~@# with unsigned 64-bit signed integer values for each component stored in order X then Y. | | Point3S8 | ''21'' | 3 bytes | 3-Component #@LinkApiDocDE2_HTML~classdecPoint3.html,point~@# with signed 8-bit signed integer values for each component stored in order X, Y then Z. | | Point3U8 | ''22'' | 3 bytes | 3-Component #@LinkApiDocDE2_HTML~classdecPoint3.html,point~@# with unsigned 8-bit signed integer values for each component stored in order X, Y then Z. | | Point3S16 | ''23'' | 6 bytes | 3-Component #@LinkApiDocDE2_HTML~classdecPoint3.html,point~@# with signed 16-bit signed integer values for each component stored in order X, Y then Z. | | Point3U16 | ''24'' | 6 bytes | 3-Component #@LinkApiDocDE2_HTML~classdecPoint3.html,point~@# with unsigned 16-bit signed integer values for each component stored in order X, Y then Z. | | Point3S32 | ''25'' | 3 bytes | 3-Component #@LinkApiDocDE2_HTML~classdecPoint3.html,point~@# with signed 32-bit signed integer values for each component stored in order X, Y then Z. | | Point3U32 | ''26'' | 3 bytes | 3-Component #@LinkApiDocDE2_HTML~classdecPoint3.html,point~@# with unsigned 32-bit signed integer values for each component stored in order X, Y then Z. | | Point3S64 | ''27'' | 3 bytes | 3-Component #@LinkApiDocDE2_HTML~classdecPoint3.html,point~@# with signed 64-bit signed integer values for each component stored in order X, Y then Z. | | Point3U64 | ''28'' | 3 bytes | 3-Component #@LinkApiDocDE2_HTML~classdecPoint3.html,point~@# with unsigned 64-bit signed integer values for each component stored in order X, Y then Z. | | Vector2F16 | ''29'' | 4 bytes | 2-Component #@LinkApiDocDE2_HTML~classdecVector2.html,vector~@# with 16-bit floating point values for each component stored in order X then Y. | | Vector2F32 | ''30'' | 8 bytes | 2-Component #@LinkApiDocDE2_HTML~classdecVector2.html,vector~@# with 32-bit floating point values for each component stored in order X then Y. | | Vector2F64 | ''31'' | 16 bytes | 2-Component #@LinkApiDocDE2_HTML~classdecVector2.html,vector~@# with 64-bit floating point values for each component stored in order X then Y. | | Vector3F16 | ''32'' | 6 bytes | 3-Component #@LinkApiDocDE2_HTML~classdecVector.html,vector~@# with 16-bit floating point values for each component stored in order X, Y then Z. | | Vector3F32 | ''33'' | 12 bytes | 3-Component #@LinkApiDocDE2_HTML~classdecVector.html,vector~@# with 32-bit floating point values for each component stored in order X, Y then Z. | | Vector3F64 | ''34'' | 24 bytes | 3-Component #@LinkApiDocDE2_HTML~classdecVector.html,vector~@# with 64-bit floating point values for each component stored in order X, Y then Z. | | QuaternionF16 | ''35'' | 8 bytes | 4-Component #@LinkApiDocDE2_HTML~classdecQuaternion.html,quaternion~@# with 16-bit floating point values for each component stored in order X, Y, Z then W. | | QuaternionF32 | ''36'' | 16 bytes | 4-Component #@LinkApiDocDE2_HTML~classdecQuaternion.html,quaternion~@# with 32-bit floating point values for each component stored in order X, Y, Z then W. | | QuaternionF64 | ''37'' | 32 bytes | 4-Component #@LinkApiDocDE2_HTML~classdecQuaternion.html,quaternion~@# with 64-bit floating point values for each component stored in order X, Y, Z then W. | ====== Reliable Communication ====== Certain commands are reliable. They are guaranteed to be delivered or else the connection is interrupted. In contrary to unreliable commands reliable commands can be delayed for up to //ReliableTimeout// (recommended 3s). If a reliable command is not acknowledged before //ReliableResendInterval// (recommended 0.5s) the sender has to resend the same command. This is repeated every //ReliableResendInterval// until the command is acknowledged or //ReliableTimeout// is reached. The receiving side has to ignore duplicate or not expected commands. To achieve this each communication side maintains a //NextSendCommand// and //NextReceiveCommand// value for each connection. This value has to be set to ''0'' upon every accepted connection request. The sender assigns the next //NextSendCommand// to the command to be reliably send. It then increments //NextSendCommand// by one. If //NextSendCommand// reaches ''65'536'' the value has to wrapped around to ''0''. Upon receiving a reliable command the receiver validates the command number against //NextReceiveCommand// using these rules: * if CommandNumber < NextReceiveCommand then valid if CommandNumber < (NextReceiveCommand + ReliableWindowSize) % 65535 * else valid if CommandNumber < NextReceiveCommand + ReliableWindowSize //ReliableWindowSize// is ''10'' and is used to prevent outdated or stray commands being processed. If the command is not accepted it has to be discarded and processing this command stops. If //CommandNumber// equals //NextReceiveCommand// the command is processed and send to the application since it is the next command expected to arrive. In this case //NextReceiveCommand// is incremented by1. If //NextReceiveCommand// reaches ''65'536'' the value has to be wrapped around to ''0''. If //CommandNumber// does not equal //NextReceiveCommand// the command has over-taken the next expected command and has to be held back until all commands in between have been received, processed and send to the application. Whenever a held back command is processed //NextReceiveCommand// has to be incremented as outlined above. Every valid received command has to be acknowledged. This tells the sending side the command has been received and no resending is required. ====== Network States ====== Clients and server can link local network states to the other side. Both client and server can send link requests to the other side. It is up to the application to decide if a link request is accepted or declined. Link requests are handled like [[#reliable_message|Reliable Messages]] in that they are send reliable and contain a message the application can use to know what kind of state to set up. The receiving application has to set up the state to link exactly the same way as the sending application did. This means the same count of values with the same [[#state_value_data_type|data types]]. If this is not the case the receiving client has to decline the state linking. This is in particular the case if the receiving application returns a //null// state. This is done if the receiving application declines linking a state for whatever reason. The sender assigns a //LinkId// to the state to link. This identifier is then used in the future to reference a linked state. Once assigned the //LinkId// can not be reused until it has been taken down. Multiple clients maintain own links for a local network state. Each connection has potentially a different //LinkId//. Hence //LInkId// are not shared across connections although the state they link are. In addition to the message delivered to the application the command contains the initial value for all values in the network state. State links can be read write or read only. The sender decided if the receiver is allowed to modify the state or not. Servers usually send read-only states to clients for all entities controlled by the game or other players. The client usually gets a read-write state for his own state he has to modify. The sender has to ignore attempts of clients trying to update states that are read-only to them. The receiver has to send back a [[#link_up|Link Up]] command if the link is accepted or a [[#link_down|Link Down]] command to reject it. As soon as a [[#link_down|Link Down]] command is send the //LinkId// in question becomes free again to be used for a new link. ====== Commands ====== Commands begin with an unsigned 8-bit integer //CommandCode// indicating the command type. Commands have no explicit length. The command content is the remaining data in the UDP package after the //CommandCode//. Network clients are required to ignore all commands with unknown //CommandCode//. The following commands are supported: ^ Command Code ^ Name ^ Protocol Version ^ | ''0'' | [[#connection_request|Connection Request]] | DNP1 | | ''1'' | [[#connection_ack|Connection Acknowledge]] | DNP1 | | ''2'' | [[#connection_close|Connection Close]] | DNP1 | | ''3'' | [[#message|Unreliable Message]] | DNP1 | | ''4'' | [[#reliable_message|Reliable Message]] | DNP1 | | ''5'' | [[#reliable_link_state|Link State]] | DNP1 | | ''6'' | [[#reliable_ack|Link Acknowledge]] | DNP1 | | ''7'' | [[#link_up|Link Up]] | DNP1 | | ''8'' | [[#link_down|Link Down]] | DNP1 | | ''9'' | [[#link_update|Link Update]] | DNP1 | ===== Connection Request ===== Send by the connecting client to connect to a server. This is the only command accepted by the server from an unconnected client. The server accepts the connection if no other connection exists with the same hostname and port combination and no other server specific rules forbid the client from connecting. If the client does not receive a [[#connect_ack|Connect Acknowledge]] command until //Connect Resend Interval// (recommended 1s) after sending the last request the command has to be resend. If //Connect Timeout// (recommended 5s) elapsed since the first request send the client has to stop resending the connect request and report connect timeout error to the user. The server has to ignore connect requests for an already connected client. The command has this format: ^ Type ^ Name ^ Description ^ | [[#data_types|Byte]] | CommandCode | Command code. Is value ''0'' | | [[#data_types|UShort]] | ProtocolCount | Count of protocols supported by the client. | | [[#data_types|UShort]][ProtocolCount] | Supported Protocols | List of protocols supported by the client. Each list entry has to be one of these values: * ''0'': Drag[en]gine Network Protocol Version 1 (DNP1) New protocols can be added in the future. Servers have to ignore unknown protocol values. | ===== Connection Ack ===== Send by the server as answer to the [[#connect_request|Connect Request]] send by a connecting client. The server accepts the connection if no other connection exists with the same hostname and port combination and no other server specific rules forbid the client from connecting. The server checks if the client supports at least one protocol the server supports too. If multiple protocols are supported the server selects the best suited protocol, typically the one with the most features. If the server rejects the client it has to forget about the connect request after sending the answer. The command has this format: ^ Type ^ Name ^ Description ^ | [[#data_types|Byte]] | Command Code | Command code. Is value ''1'' | | [[#data_types|Byte]] | Result Code | Decision of the server. Has to be one of these values: * ''0'': Connection accepted. * ''1'': Connection rejected due to technical reasons or restrictions imposed on the client. * ''2'': Connection rejected because the client and server have no common protocol to speak. More result codes can be added in the future. All unknown result codes have to be treated is if //ResultCode// had been ''1''. | | [[#data_types|UInt16]] | Protocol | Protocol selected by the server. This value is only present if //Result Code// has the value ''0''. | ===== Close Connection ===== Send by the client before closing connection to server. Send by server before disconnecting client unless this is the client having send a close connection. No acknowledge is send. This command is best effort. If the command is lost client and server have to figure out connection loss using other means. The command has this format: ^ Type ^ Name ^ Description ^ | [[#data_types|Byte]] | CommandCode | Command code. Is value ''2'' | ===== Message ===== Send by client or server or server to client. used to send an unreliable message to the other communication partner. Unreliable messages are used for frequently send messages. They are not acknowledged and thus can be possibly lost. Furthermore they can be delivered to the application in any order. Applications have to use unreliable messages only if the loss of information can be compensated, for example by receiving another update in regular intervals. Messages should be of short length. No explicit fragmenting of message content is done. This is left for the underlying communication channel. If a message is not fully received it has to be considered lost. The exact message size fitting into one communication channel package depends on the communication channel and is often not detectable. As a rule of thumb on IPv4 a package size of 1200 (conservative 540) can be expected to be transmitted in one package. The command has this format: ^ Type ^ Name ^ Description ^ | [[#data_types|Byte]] | CommandCode | Command code. Is value ''3'' | | [[#data_types|Byte]][*] | Data | Message data. The length of the data is the remaining count of bytes in the UDP package. | ===== Reliable message ===== Send by client or server or from server to client. Used to send a reliable message to the other communication partner. Reliable messages are used for infrequent send messages of high importance. They are acknowledged and guaranteed to be received. Furthermore they are guaranteed to be delivered to the application in the order they have been send. Applications have to use reliable messages only if loss of information can not be compensated. Messages should be of short length. No explicit fragmenting of message content is done. This is left for the underlying communication channel. If a message is not fully received it has to be considered lost. The exact message size fitting into one communication channel package depends on the communication channel and is often not detectable. As a rule of thumb on IPv4 a package size of 1200 (conservative 540) can be expected to be transmitted in one package. The command has this format: ^ Type ^ Name ^ Description ^ | [[#data_types|Byte]] | CommandCode | Command code. Is value ''4'' | | [[#data_types|UInt16]] | Number | Command sequence number. See [[#reliable_communication|Reliable Communication]]. | | [[#data_types|Byte]][*] | Data | Message data. The length of the data is the remaining count of bytes in the UDP package. | ===== Link state ===== Send by client to server or server to client. Used to [[#network_state|link a local network state]] to the other side. Both client and server can send link requests to the other side. It is up to the application to decide if a link request is accepted or declined. Messages should be of short length. No explicit fragmenting of message content is done. This is left for the underlying communication channel. If a message is not fully received it has to be considered lost. The exact message size fitting into one communication channel package depends on the communication channel and is often not detectable. As a rule of thumb on IPv4 a package size of 1200 (conservative 540) can be expected to be transmitted in one package. This size is reduced by the count of value data send along the message. The command has this format: ^ Type ^ Name ^ Description ^ | [[#data_types|Byte]] | CommandCode | Command code. Is value ''5'' | | [[#data_types|UInt16]] | Number | Command sequence number. See [[#reliable_communication|Reliable Communication]]. | | [[#data_types|UInt16]] | LinkId | Identifier of link. | | [[#data_types|Byte]] | Flags | Flags type value. Flags have to be values from this list: * ''1'': The state link is read only. | | [[#data_types|UInt16]] | MessageLength | Length of message in bytes. | | [[#data_types|Byte]][MessageLength] | Message | Message data. | | [[#data_types|UInt16]] | ValueCount | Count of values. | | ValueData[ValueCount] | Values | Initial value of all values in the state. | ValueData has this format: ^ Type ^ Name ^ Description ^ | [[#data_types|Byte]] | Type | [[#state_value_data_type|Data Type]] of value. | | * | Data | Initial value. The count of bytes required by //Data// is defined by the //Length// attribute in [[#state_value_data_types|State Value Data Type]]. | ===== Reliable ack ===== Send by client to server or from server to client. Used to acknowledge receiving a reliable message to the other communication partner. The command has this format: ^ Type ^ Name ^ Description ^ | [[#data_types|Byte]] | CommandCode | Command code. Is value ''6'' | | [[#data_types|UInt16]] | Number | Command sequence number. See [[#reliable_communication|Reliable Communication]]. | | [[#data_types|Byte]] | ResultCode | Result of operation. Has to be one of these values: * ''0'': Command received successfully. * ''1'': Receiving command failed. Used if the receiver would like sender to resend the message. | ===== Link up ===== Send by client to server or server to client. Used to accept a [[#link_state|Link State]] command. Only after receiving a Link Up command the client is allowed to send state updates if the state in question is read-write to the client. The sender can not assign //LinkId// to any other link until sending a [[#link_down|Link Down]] command to free a //LinkId//. The command has this format: ^ Type ^ Name ^ Description ^ | [[#data_types|Byte]] | CommandCode | Command code. Is value ''7'' | | [[#data_types|UInt16]] | LinkId | Identifier of link. | ===== Link down ===== Send by client to server or server to client. Used to tear down a [[#link_state|Link State]]. Once torn down the sender can reuse the //LinkId// for another link. It is recommended to keep assigning new //LinkId// until the value range is exhausted before reusing //LinkId//. The command has this format: ^ Type ^ Name ^ Description ^ | [[#data_types|Byte]] | CommandCode | Command code. Is value ''8'' | | [[#data_types|UInt16]] | LinkId | Identifier of link. | ===== Link update ===== Send by client to server or server to client. Used to update a remote state. Client or server is only allowed to send a state update if it has a read-write state. This is the case if the client or server linked the state to other sides or it received a state with read-write capability. The command contains one or more values to update which do not have to be of the same state. The sender is not required to update all values with one command send. The sender decides how many values it includes in the command and when it updates the values. In general the sender has to update state changes as fast as possible. The data type range and precision information assigned to values has to be used to send updates only if necessary. For example a floating point value with 0.1 precision should only send an update if the value set by the application deviates more than 0.1 from the last known value. The application sets the precision according to needs. Optionally the sender can update state values if they have recently changed but not more than their precision requires. This can be used as a kind of "keep-alive". In this case //LinkCount// is ''0''. If the receives values change enough (according to the same rules as sending them) the application has to be send notification for each changed value. The command has this format: ^ Type ^ Name ^ Description ^ | [[#data_types|Byte]] | CommandCode | Command code. Is value ''9'' | | [[#data_types|UInt16]] | LinkCount | Count of links to update. | | LinkData[LinkCount] | Links | Link data | LinkData has this format: ^ Type ^ Name ^ Description ^ | [[#data_types|UInt16]] | LinkId | Identifier of link. | | [[#data_types|UInt16]] | ValueCount | Count of values. | | ValueData[ValueCount] | Values | Updated values. | ValueData has this format: ^ Type ^ Name ^ Description ^ | [[#data_types|Byte]] | Type | [[#state_value_data_type|Data Type]] of value. | | * | Data | Updated value. The count of bytes required by //Data// is defined by the //Length// attribute in [[#state_value_data_types|State Value Data Type]]. |