MessagePack extensions for the datetime and interval types (#3077)

* Add datetime and interval types
* Update translation

Author: xuniq

doc/book/box/data_model.rst

@@ -304,7 +304,7 @@ In this case, the number will be stored as `cdata`_.
 
 ..  _index-box_decimal:
 
-**decimal**. The Tarantool decimal type is stored as a MsgPack ext (Extension).
+**decimal**. The Tarantool decimal type is stored as a :ref:`MsgPack ext <msgpack_ext-decimal>` (Extension).
 Values with the decimal type are not floating-point values although
 they may contain decimal points.
 They are exact with up to 38 digits of precision.
@@ -315,7 +315,7 @@ Example: a value returned by a function in the :ref:`decimal <decimal>` module.
 **datetime**. Introduced in :tarantool-release:`2.10.0`.
 The Tarantool ``datetime`` type facilitates operations with date and time,
 accounting for leap years or the varying number of days in a month.
-It is stored as a MsgPack ext (Extension).
+It is stored as a :ref:`MsgPack ext <msgpack_ext-datetime>` (Extension).
 Operations with this data type use code from `c-dt <https://github.com/tarantool/c-dt>`_, a third-party library.
 
 For more information, see :doc:`Module datetime </reference/reference_lua/datetime>`.
@@ -326,7 +326,7 @@ For more information, see :doc:`Module datetime </reference/reference_lua/dateti
 The Tarantool ``interval`` type represents periods of time.
 They can be added to or subtracted from ``datetime`` values or each other.
 Operations with this data type use code from `c-dt <https://github.com/tarantool/c-dt>`_, a third-party library.
-
+The type is stored as a :ref:`MsgPack ext <msgpack_ext-interval>` (Extension).
 For more information, see :doc:`Module datetime </reference/reference_lua/datetime>`.
 
 ..  _index-box_string:
@@ -356,7 +356,7 @@ Example: ``"\65 \66 \67"``.
 **uuid**. The Tarantool uuid type is used for
 :ref:`Universally Unique Identifiers <uuid-module>`.
 Since version :doc:`2.4.1 </release/2.4.1>` Tarantool stores
-``uuid`` values as a MsgPack ext (Extension).
+``uuid`` values as a :ref:`MsgPack ext <msgpack_ext-uuid>` (Extension).
 
 Example: ``64d22e4d-ac92-4a23-899a-e5934af5479``.
 

doc/dev_guide/internals/msgpack_extensions.rst

@@ -1,16 +1,17 @@
-.. _internals-msgpack_ext:
-
-.. _msgpack_ext-decimal:
+..  _internals-msgpack_ext:
 
 ----------------------------
 MessagePack extensions
 ----------------------------
 
 Tarantool uses predefined MessagePack extension types to represent some
-of the special values. Extension types include ``MP_DECIMAL``, ``MP_UUID``
-and ``MP_ERROR``. These types require special attention from the connector
-developers, as they must be treated separately from the default MessagePack
-types, and correctly mapped to programming language types.
+of the special values. Extension types include ``MP_DECIMAL``, ``MP_UUID``,
+``MP_ERROR``, ``MP_DATETIME``, and ``MP_INTERVAL``.
+These types require special attention from the connector developers,
+as they must be treated separately from the default MessagePack types,
+and correctly mapped to programming language types.
+
+..  _msgpack_ext-decimal:
 
 *******************************
 The DECIMAL type
@@ -19,19 +20,19 @@ The DECIMAL type
 The MessagePack EXT type ``MP_EXT`` together with the extension type
 ``MP_DECIMAL`` is a header for values of the DECIMAL type.
 
-MP_DECIMAL is 1.
+``MP_DECIMAL`` type is 1.
 
-`MessagePack spec <https://github.com/msgpack/msgpack/blob/master/spec.md>`_
+`MessagePack specification <https://github.com/msgpack/msgpack/blob/master/spec.md>`_
 defines two kinds of types:
 
-* ``fixext 1/2/4/8/16`` types have fixed length so the length is not encoded explicitly;
+* ``fixext 1/2/4/8/16`` types have fixed length so the length is not encoded explicitly.
 * ``ext 8/16/32`` types require the data length to be encoded.
 
 ``MP_EXP`` + optional ``length`` imply using one of these types.
 
 The decimal MessagePack representation looks like this:
 
-.. code-block:: none
+..  code-block:: none
 
     +--------+-------------------+------------+===============+
     | MP_EXT | length (optional) | MP_DECIMAL | PackedDecimal |
@@ -42,7 +43,7 @@ Here ``length`` is the length of ``PackedDecimal`` field, and it is of type
 
 ``PackedDecimal`` has the following structure:
 
-.. code-block:: none
+..  code-block:: none
 
      <--- length bytes -->
     +-------+=============+
@@ -61,7 +62,7 @@ The rightmost digit in the array is the least significant.
 The first byte of the ``BCD`` array contains the first digit of the number,
 represented as follows:
 
-.. code-block:: none
+..  code-block:: none
 
     |  4 bits           |  4 bits           |
        = 0x                = the 1st digit
@@ -70,7 +71,7 @@ represented as follows:
 The last byte of the ``BCD`` array contains the last digit of the number and the
 final ``nibble``, represented as follows:
 
-.. code-block:: none
+..  code-block:: none
 
     |  4 bits           |  4 bits           |
        = the last digit    = nibble
@@ -84,21 +85,21 @@ The final ``nibble`` represents the number's sign:
 
 The decimal ``-12.34`` will be encoded as ``0xd6,0x01,0x02,0x01,0x23,0x4d``:
 
-.. code-block:: none
+..  code-block:: none
 
     |MP_EXT (fixext 4) | MP_DECIMAL | scale |  1   |  2,3 |  4 (minus) |
     |       0xd6       |    0x01    | 0x02  | 0x01 | 0x23 | 0x4d       |
 
 The decimal 0.000000000000000000000000000000000010
 will be encoded as ``0xc7,0x03,0x01,0x24,0x01,0x0c``:
 
-.. code-block:: none
+..  code-block:: none
 
     | MP_EXT (ext 8) | length | MP_DECIMAL | scale |  1   | 0 (plus) |
     |      0xc7      |  0x03  |    0x01    | 0x24  | 0x01 | 0x0c     |
 
 
-.. _msgpack_ext-uuid:
+..  _msgpack_ext-uuid:
 
 **********************************
 The UUID type
@@ -107,31 +108,35 @@ The UUID type
 The MessagePack EXT type ``MP_EXT`` together with the extension type
 ``MP_UUID`` for values of the UUID type. Since version :doc:`2.4.1 </release/2.4.1>`.
 
-MP_UUID is 2.
+``MP_UUID`` type is 2.
 
-The `MessagePack spec <https://github.com/msgpack/msgpack/blob/master/spec.md>`_
-defines ``d8`` to mean fixext with size 16, and a uuid's size is always 16.
-So the uuid MessagePack representation looks like this:
+The `MessagePack specification <https://github.com/msgpack/msgpack/blob/master/spec.md>`_
+defines ``d8`` to mean ``fixext`` with size 16, and a UUID's size is always 16.
+So the UUID MessagePack representation looks like this:
 
-.. code-block:: none
+..  code-block:: none
 
     +--------+------------+-----------------+
     | MP_EXT | MP_UUID    | UuidValue       |
     | = d8   | = 2        | = 16-byte value |
     +--------+------------+-----------------+
 
-
 The 16-byte value has 2 digits per byte.
-Typically it consists of 11 fields, which are encoded as big endian
-unsigned integers in the following order: time_low (4 bytes), time_mid
-(2 bytes), time_hi_and_version (2 bytes), clock_seq_hi_and_reserved (1
-byte), clock_seq_low (1 byte), node[0], ..., node[5] (1 byte each).
+Typically, it consists of 11 fields, which are encoded as big-endian
+unsigned integers in the following order:
+
+*   ``time_low`` (4 bytes)
+*   ``time_mid`` (2 bytes)
+*   ``time_hi_and_version`` (2 bytes)
+*   ``clock_seq_hi_and_reserved`` (1 byte)
+*   ``clock_seq_low`` (1 byte)
+*   ``node[0]``, ..., ``node[5]`` (1 byte each)
 
 Some of the functions in :ref:`Module uuid <uuid-module>` can produce values
 which are compatible with the UUID data type.
 For example, after
 
-.. code-block:: none
+..  code-block:: none
 
     uuid = require('uuid')
     box.schema.space.create('t')
@@ -141,26 +146,28 @@ For example, after
 
 a peek at the server response packet will show that it contains
 
-.. code-block:: none
+..  code-block:: none
 
     d8 02 f6 42 3b df b4 9e 49 13 b3 61 07 40 c9 70 2e 4b
 
-.. _msgpack_ext-error:
+..  _msgpack_ext-error:
 
 ****************************************************
 The ERROR type
 ****************************************************
 
 Since version :doc:`2.4.1 </release/2.4.1>`, responses for errors have extra information
-following what was described in :ref:`Box protocol -- responses for errors
-<box_protocol-responses_error>`.
+following what was described in
+:ref:`Box protocol -- responses for errors <box_protocol-responses_error>`.
 This is a "compatible" enhancement, because clients that expect old-style
 server responses should ignore map components that they do not recognize.
 Notice, however, that there has been a renaming of a constant:
-formerly IPROTO_ERROR in ./box/iproto_constants.h was 0x31,
-now IPROTO_ERROR is 0x52 and IPROTO_ERROR_24 is 0x31.
+formerly ``IPROTO_ERROR`` in :file:`./box/iproto_constants.h` was ``0x31``,
+now ``IPROTO_ERROR`` is ``0x52`` and ``IPROTO_ERROR_24`` is ``0x31``.
 
-.. code-block:: none
+``MP_ERROR`` type is 3.
+
+..  code-block:: none
 
     ++=========================+============================+
     ||                         |                            |
@@ -170,8 +177,8 @@ now IPROTO_ERROR is 0x52 and IPROTO_ERROR_24 is 0x31.
     ++=========================+============================+
                             MP_MAP
 
-The extra information, most of which is also in :doc:`error object
-</reference/reference_lua/box_error/new>` fields, is:
+The extra information, most of which is also in
+:doc:`error object </reference/reference_lua/box_error/new>` fields, is:
 
 ``MP_ERROR_TYPE`` (0x00) (MP_STR) Type that implies source, as in :samp:`{error_object}.base_type`, for example "ClientError".
 
@@ -203,7 +210,7 @@ For example, in version 2.4.1 or later, if we try to create a duplicate space wi
 ``conn:eval([[box.schema.space.create('_space');]])`` |br|
 the server response will look like this:
 
-.. code-block:: none
+..  code-block:: none
 
     ce 00 00 00 88                  MP_UINT = HEADER + BODY SIZE
     83                              MP_MAP, size 3 (i.e. 3 items in header)
@@ -233,3 +240,141 @@ the server response will look like this:
             00                              MP_UINT = error number
             05                              MP_ERROR_ERRCODE
             0a                              MP_UINT = eror code ER_SPACE_EXISTS
+
+
+..  _msgpack_ext-datetime:
+
+**********************************
+The DATETIME type
+**********************************
+
+Since version :doc:`2.10.0 </release/2.10.0>`.
+The MessagePack EXT type ``MP_EXT`` together with the extension type
+``MP_DATETIME`` is a header for values of the DATETIME type.
+It creates a container with a payload of 8 or 16 bytes.
+
+``MP_DATETIME`` type is 4.
+
+The `MessagePack specification <https://github.com/msgpack/msgpack/blob/master/spec.md>`_
+defines ``d7`` to mean ``fixext`` with size 8 or ``d8`` to mean ``fixext`` with size 16.
+
+So the datetime MessagePack representation looks like this:
+
+..  code-block:: none
+
+    +---------+----------------+==========+-----------------+
+    | MP_EXT  | MP_DATETIME    | seconds  | nsec; tzoffset; |
+    | = d7/d8 | = 4            |          | tzindex;        |
+    +---------+----------------+==========+-----------------+
+
+MessagePack data contains:
+
+*   Seconds (8 bytes) as an unencoded 64-bit signed integer stored in the little-endian order.
+
+*   The optional fields (8 bytes), if any of them have a non-zero value.
+    The fields include ``nsec``, ``tzoffset``, and ``tzindex`` packed in the little-endian order.
+
+For more information about the datetime type, see :ref:`datetime field type details <index-box_datetime>`
+and :doc:`reference for the datetime module </reference/reference_lua/datetime>`.
+
+..  _msgpack_ext-interval:
+
+**********************************
+The INTERVAL type
+**********************************
+
+Since version :doc:`2.10.0 </release/2.10.0>`.
+The MessagePack EXT type ``MP_EXT`` together with the extension type
+``MP_INTERVAL`` is a header for values of the INTERVAL type.
+
+``MP_INTERVAL`` type is 6.
+
+The interval is saved as a variant of a map with a predefined number of known attribute names.
+If some attributes are undefined, they are omitted from the generated payload.
+
+The interval MessagePack representation looks like this:
+
+..  code-block:: none
+
+    +--------+-------------------------+-------------+----------------+
+    | MP_EXT | Size of packed interval | MP_INTERVAL | PackedInterval |
+    +--------+-------------------------+-------------+----------------+
+
+Packed interval consists of:
+
+*   Packed number of non-zero fields.
+*   Packed non-null fields.
+
+Each packed field has the following structure:
+
+..  code-block:: none
+
+    +----------+=====================+
+    | field ID |     field value     |
+    +----------+=====================+
+
+The number of defined (non-null) fields can be zero.
+In this case, the packed interval will be encoded as integer 0.
+
+List of the field IDs:
+
+*   0 -- year
+*   1 -- month
+*   2 -- week
+*   3 -- day
+*   4 -- hour
+*   5 -- minute
+*   6 -- second
+*   7 -- nanosecond
+*   8 -- adjust
+
+**Example**
+
+Interval value ``1 years, 200 months, -77 days`` is encoded in the following way:
+
+..  code-block:: tarantoolsession
+
+    tarantool> I = datetime.interval.new{year = 1, month = 200, day = -77}
+    ---
+    ...
+
+    tarantool> I
+    ---
+    - +1 years, 200 months, -77 days
+    ...
+
+    tarantool> M = msgpack.encode(I)
+    ---
+    ...
+
+    tarantool> M
+    ---
+    - !!binary xwsGBAABAczIA9CzCAE=
+    ...
+
+    tarantool> tohex = function(s) return (s:gsub('.', function(c) return string.format('%02X ', string.byte(c)) end)) end
+    ---
+    ...
+
+    tarantool> tohex(M)
+    ---
+    - 'C7 0B 06 04 00 01 01 CC C8 03 D0 B3 08 01 '
+    ...
+
+Where:
+
+*   C7 -- MP_EXT
+*   0B -- size of a packed interval value (11 bytes)
+*   06 -- MP_INTERVAL type
+*   04 -- number of defined fields
+*   00 -- field ID (year)
+*   01 -- packed value ``1``
+*   01 -- field ID (month)
+*   CCC8 -- packed value ``200``
+*   03 -- field ID (day)
+*   D0B3 -- packed value ``-77``
+*   08 -- field ID (adjust)
+*   01 -- packed value ``1`` (DT_LIMIT)
+
+For more information about the interval type, see :ref:`interval field type details <index-box_interval>`
+and :doc:`description of the datetime module </reference/reference_lua/datetime>`.

locale/ru/LC_MESSAGES/book/box/data_model.po

@@ -531,12 +531,12 @@ msgstr ""
 msgid ""
 "**uuid**. The Tarantool uuid type is used for :ref:`Universally Unique "
 "Identifiers <uuid-module>`. Since version :doc:`2.4.1 </release/2.4.1>` "
-"Tarantool stores ``uuid`` values as a MsgPack ext (Extension)."
+"Tarantool stores ``uuid`` values as a :ref:`MsgPack ext <msgpack_ext-uuid>` (Extension)."
 msgstr ""
 "**uuid**. Тип uuid в Tarantool используется для :ref:`универсальных "
 "уникальных идентификаторов (UUID) <uuid-module>`. Начиная с версии "
-":doc:`2.4.1 </release/2.4.1>` Tarantool хранит значения ``uuid`` в формате "
-"MsgPack ext (Extension)."
+":doc:`2.4.1 </release/2.4.1>`, Tarantool хранит значения ``uuid`` в формате "
+":ref:`MsgPack ext <msgpack_ext-uuid>` (Extension)."
 
 msgid "Example: ``64d22e4d-ac92-4a23-899a-e5934af5479``."
 msgstr "Пример: ``64d22e4d-ac92-4a23-899a-e5934af5479``."