qemu-block
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Qemu-block] [PATCH] docs/interop/bitmaps: rewrite and modernize doc


From: John Snow
Subject: Re: [Qemu-block] [PATCH] docs/interop/bitmaps: rewrite and modernize doc
Date: Mon, 22 Apr 2019 14:59:07 -0400
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1


On 4/18/19 12:38 PM, Vladimir Sementsov-Ogievskiy wrote:
> 18.04.2019 3:14, John Snow wrote:
>> This just about rewrites the entirety of the bitmaps.rst document to
>> make it consistent with the 4.0 release. I have added new features seen
>> in the 4.0 release, as well as tried to clarify some points that keep
>> coming up when discussing this feature both in-house and upstream.
>>
>> Yes, it's a lot longer, mostly due to examples. I get a bit chatty.
>> I could use a good editor to help reign in my chattiness.
>>
>> It does not yet cover pull backups or migration details, but I intend to
>> keep extending this document to cover those cases.
>>
>> Please try compiling it with sphinx and look at the rendered output, I
>> don't have hosting to share my copy at present. I think this new layout
>> reads nicer in the HTML format than the old one did, at the expense of
>> looking less readable in the source tree itself (though not completely
>> unmanagable. We did decide to convert it from Markdown to ReST, after
>> all, so I am going all-in on ReST.)
>>
>> Signed-off-by: John Snow <address@hidden>
>> ---
>>   docs/interop/bitmaps.rst | 1499 ++++++++++++++++++++++++++++++--------
>>   Makefile                 |    2 +-
>>   2 files changed, 1192 insertions(+), 309 deletions(-)
>>
>> diff --git a/docs/interop/bitmaps.rst b/docs/interop/bitmaps.rst
>> index 7bcfe7f461..a39d1fc871 100644
>> --- a/docs/interop/bitmaps.rst
>> +++ b/docs/interop/bitmaps.rst
> 
> you may want to update copyright date at the beginning of the file
> 

Ah, I guess so. I don't really know how copyright works anyway :)

>> @@ -9,128 +9,481 @@
>>   Dirty Bitmaps and Incremental Backup
>>   ====================================
>>   
>> --  Dirty Bitmaps are objects that track which data needs to be backed up
>> -   for the next incremental backup.
>> +Dirty Bitmaps are in-memory objects that track writes to block devices. 
>> They can
>> +be used in conjunction with various block job operations to perform 
>> incremental
>> +or differential backup regimens.
>>   
>> --  Dirty bitmaps can be created at any time and attached to any node
>> -   (not just complete drives).
>> +This document explains the conceptual mechanisms, as well as up-to-date,
>> +complete and comprehensive documentation on the API to manipulate them.
>> +(Hopefully, the "why", "what", and "how".)
>> +
>> +The intended audience for this document is developers who are adding QEMU 
>> backup
>> +features to management applications, or power users who run and administer 
>> QEMU
>> +directly via QMP.
>>   
>>   .. contents::
>>   
>> +Overview
>> +--------
>> +
>> +Bitmaps are bit vectors where each '1' bit in the vector indicates a 
>> modified
>> +("dirty") segment of the corresponding block device. The size of the segment
>> +that is tracked is the granularity of the bitmap. If the granularity of a 
>> bitmap
>> +is 64K, each '1' bit means that an entire 64K region changed in some way.
> 
> hm not exactly. Сonversely, if we change not the entire region but only on 
> byte of it,
> corresponding bit in the bitmap would be set..
> 

Ah, yeah, I worded this oddly. I meant to say that taken as a whole, a
64k region changed in some way (possibly by as little as just one bit.)

I'll fix this.

>> +
>> +Smaller granularities mean more accurate tracking of modified disk data, but
>> +requires more computational overhead and larger bitmap sizes. Larger
>> +granularities mean smaller bitmap sizes, but less targeted backups.
>> +
>> +The size of a bitmap (in bytes) can be computed as such:
>> +    ``size`` = ((``image_size`` / ``granularity``) / 8)
> 
> both divisions should round up
> 

Will clarify. It's also not quite true because of the hierarchical
storage requirements too; this is really the size on disk ... but it's a
useful heuristic for people to know, anyway.

"It's about 1MB per 512GB, until you adjust tuning."

>> +
>> +e.g. the size of a 64KiB granularity bitmap on a 2TiB image is:
>> +    ``size`` = ((2147483648K / 64K) / 8)
>> +         = 4194304B = 4MiB.
>> +
>> +QEMU uses these bitmaps when making incremental backups to know which
>> +sections of the file to copy out. They are not enabled by default and
>> +must be explicitly added in order to begin tracking writes.
>> +
>> +Bitmaps can be created at any time and can be attached to any
>> +arbitrary block node in the storage graph, but are most useful
>> +conceptually when attached to the root node attached to the guest's
>> +storage device model.
>> +
>> +(Which is a really chatty way of saying: It's likely most useful to
>> +track the guest's writes to disk, but you could theoretically track
>> +things like qcow2 metadata changes by attaching the bitmap elsewhere
>> +in the storage graph.)
>> +
>> +QEMU supports persisting these bitmaps to disk via the qcow2 image format.
>> +Bitmaps which are stored or loaded in this way are called "persistent", 
>> whereas
>> +bitmaps that are not are called "transient".
>> +
>> +QEMU also supports the migration of both transient bitmaps (tracking any
>> +arbitrary image format) or persistent bitaps (qcow2) via live migration.
> 
> s/bitaps/bitmaps
> 
> not sure it should be mentioned: only named bitmaps are migrated.
> 

Since anonymous bitmaps ought to be invisible from the QMP api, it's
probably only worth a quick mention in the migration section I intend to
write. It's useful information for QEMU developers, but not really
users, I think.

>> +
>> +Supported Image Formats
>> +-----------------------
>> +
>> +QEMU supports all documented features below on the qcow2 image format.
>> +
>> +However, qcow2 is only strictly necessary for the persistence feature, which
>> +writes bitmap data to disk upon close. If persistence is not required for a
>> +specific use case, all bitmap features excepting persistence are available
>> +for any arbitrary image format.
>> +
>> +For example, Dirty Bitmaps can be combined with the 'raw' image format,
>> +but any changes to the bitmap will be discarded upon exit.
>> +
>> +.. warning:: Transient bitmaps will not be saved on QEMU exit! Persistent
>> +             bitmaps are available only on qcow2 images.
>> +
>>   Dirty Bitmap Names
>>   ------------------
>>   
>> --  A dirty bitmap's name is unique to the node, but bitmaps attached to
>> -   different nodes can share the same name.
>> +Bitmap objects need a method to reference them in the API. All API-created 
>> and
>> +managed bitmaps have a human-readable name chosen by the user at creation 
>> time.
>>   
>> --  Dirty bitmaps created for internal use by QEMU may be anonymous and
>> -   have no name, but any user-created bitmaps must have a name. There
>> -   can be any number of anonymous bitmaps per node.
>> +-  A bitmap's name is unique to the node, but bitmaps attached to different
>> +   nodes can share the same name. Therefore, all bitmaps are addressed via 
>> their
>> +   (node, name) pair.
>>   
>> --  The name of a user-created bitmap must not be empty ("").
>> +-  The name of a user-created bitmap cannot be empty ("").
>>   
>> -Bitmap Modes
>> -------------
>> +-  Transient bitmaps can have JSON unicode names that are effectively not 
>> length
>> +   limited. (QMP protocol may restrict messages to less than 64MiB.)
>>   
>> --  A bitmap can be "frozen," which means that it is currently in-use by
>> -   a backup operation and cannot be deleted, renamed, written to, reset,
>> -   etc.
>> +-  Persistent storage formats may impose their own requirements on bitmap 
>> names
>> +   and namespaces. Presently, only qcow2 supports persistent bitmaps. See
>> +   docs/interop/qcow2.txt for more details on restrictions. Notably:
>>   
>> --  The normal operating mode for a bitmap is "active."
>> +   -  qcow2 bitmap names are limited to between 1 and 1023 bytes long.
>> +
>> +   -  No two bitmaps saved to the same qcow2 file may share the same name.
>> +
>> +-  QEMU occasionally uses bitmaps for internal use which have no name. They 
>> are
>> +   hidden from API query calls, cannot be manipulated by the external API, 
>> and
>> +   are never persistent.
> 
> and are not migrated
> 

OK

>> +
>> +Bitmap Status
>> +-------------
>> +
>> +Dirty Bitmap objects can be queried with the QMP command `query-block
>> +<qemu-qmp-ref.html#index-query_002dblock>`_, and are visible via the
>> +`BlockDirtyInfo <qemu-qmp-ref.html#index-BlockDirtyInfo>`_ QAPI structure.
>> +
>> +This struct shows the name, granularity, and dirty byte count for each 
>> bitmap.
>> +Additionally, it shows several boolean status indicators:
>> +
>> +- ``recording``: This bitmap is recording guest writes.
> 
> may be not guest writes, but anything like qcow2 metadata, if bitmap is 
> attached not to root.
> 

Right. I was trying to find a way to succinctly say "writes originating
from this node or higher" ... it's kind of a subtle point, especially
when this document takes no effort to explain what the storage graph is.

(Maybe that's the next document to write?)

> 
>> +- ``busy``: This bitmap is in-use by an operation.
>> +- ``persistent``: This bitmap is a persistent type.
>> +- ``inconsistent``: This bitmap is corrupted and cannot be used.
>> +
>> +The ``+busy`` status prohibits you from deleting, clearing, or otherwise
>> +modifying a bitmap, and happens when the bitmap is being used for a backup
>> +operation or is in the process of being loaded from a migration. Many of the
>> +commands documented below will refuse to work on such bitmaps.
>> +
>> +There is also a deprecated
>> +"`DirtyBitmapStatus <qemu-qmp-ref.html#index-DirtyBitmapStatus>`_" field. a
>> +bitmap historically had five visible states:
>> +
>> +   #. ``Frozen``: This bitmap is currently in-use by an operation and is
>> +      immutable. It can't be deleted, renamed, reset, etc.
>> +
>> +      (This is now ``+busy``.)
>> +
>> +   #. ``Disabled``: This bitmap is not recording new writes from the guest.
>> +
>> +      (This is now ``-recording -busy``.)
>> +
>> +   #. ``Active``: This bitmap is recording new writes from the guest.
>> +
>> +      (This is now ``+recording -busy``.)
>> +
>> +   #. ``Locked``: This bitmap is in-use by an operation, and is immutable.
>> +      The difference from "Frozen" was primarily implementation details.
>> +
>> +      (This is now ``+busy``.)
>> +
>> +   #. ``Inconsistent``: This persistent bitmap was not saved to disk 
>> correctly,
>> +      and can no longer be used. It remains in memory to serve as an
>> +      indicator of failure.
>> +
>> +      (This is now ``+inconsistent``.)
>> +
>> +These states are directly replaced by the status indicators and should
>> +not be used. The difference between ``Frozen`` and ``Locked`` is an
>> +implementation detail and should not be relevant to external users.
>>   
>>   Basic QMP Usage
>>   ---------------
>>   
>> +The primary interface to manipulating bitmap objects is via the QMP
>> +interface. If you are not familiar, see docs/interop/qmp-intro.txt for a 
>> broad
>> +overview, and `qemu-qmp-ref <qemu-qmp-ref.html>`_ for a full reference of 
>> all
>> +QMP commands.
>> +
>>   Supported Commands
>>   ~~~~~~~~~~~~~~~~~~
>>   
>> +There are six primary bitmap-management API commands:
>> +
>>   - ``block-dirty-bitmap-add``
>>   - ``block-dirty-bitmap-remove``
>>   - ``block-dirty-bitmap-clear``
>> +- ``block-dirty-bitmap-disable``
>> +- ``block-dirty-bitmap-enable``
>> +- ``block-dirty-bitmap-merge``
>>   
>> -Creation
>> -~~~~~~~~
>> +And one related query command:
>>   
>> --  To create a new bitmap, enabled, on the drive with id=drive0:
>> +- ``query-block``
>>   
>> -.. code:: json
>> +Creation: block-dirty-bitmap-add
>> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>   
>> -    { "execute": "block-dirty-bitmap-add",
>> -      "arguments": {
>> -        "node": "drive0",
>> -        "name": "bitmap0"
>> -      }
>> -    }
>> +`block-dirty-bitmap-add
>> +<qemu-qmp-ref.html#index-block_002ddirty_002dbitmap_002dadd>`_:
>> +
>> +Creates a new bitmap that tracks writes to the specified node. granularity,
>> +persistence, and recording state can be adjusted at creation time.
>> +
>> +.. admonition:: Example
>> +
>> + to create a new, actively recording persistent bitmap:
>> +
>> + .. code:: json
>> +
>> +  -> { "execute": "block-dirty-bitmap-add",
>> +       "arguments": {
>> +         "node": "drive0",
>> +         "name": "bitmap0",
>> +         "persistent": true,
>> +       }
>> +     }
>> +
>> +  <- { "return": {} }
>>   
>>   -  This bitmap will have a default granularity that matches the cluster
>>      size of its associated drive, if available, clamped to between [4KiB,
>>      64KiB]. The current default for qcow2 is 64KiB.
>>   
>> --  To create a new bitmap that tracks changes in 32KiB segments:
>> +.. admonition:: Example
>>   
>> -.. code:: json
>> + To create a new, disabled (``-recording``), transient bitmap that tracks
>> + changes in 32KiB segments:
>>   
>> -    { "execute": "block-dirty-bitmap-add",
>> -      "arguments": {
>> -        "node": "drive0",
>> -        "name": "bitmap0",
>> -        "granularity": 32768
>> -      }
>> -    }
>> + .. code:: json
>>   
>> -Deletion
>> -~~~~~~~~
>> +  -> { "execute": "block-dirty-bitmap-add",
>> +       "arguments": {
>> +         "node": "drive0",
>> +         "name": "bitmap1",
>> +         "granularity": 32768,
>> +         "disabled": true
>> +       }
>> +     }
>>   
>> --  Bitmaps that are frozen cannot be deleted.
>> +  <- { "return": {} }
>>   
>> --  Deleting the bitmap does not impact any other bitmaps attached to the
>> +Deletion: block-dirty-bitmap-remove
>> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> +
>> +`block-dirty-bitmap-remove
>> +<qemu-qmp-ref.html#index-block_002ddirty_002dbitmap_002dremove>`_:
>> +
>> +Deletes a bitmap. Bitmaps that are ``+busy`` cannot be removed.
>> +
>> +-  Deleting a bitmap does not impact any other bitmaps attached to the
>>      same node, nor does it affect any backups already created from this
>> -   node.
>> +   bitmap or node.
>>   
>>   -  Because bitmaps are only unique to the node to which they are
>>      attached, you must specify the node/drive name here, too.
>>   
>> -.. code:: json
>> +-  Deleting a persistent bitmap will remove it from the qcow2 file.
>>   
>> -    { "execute": "block-dirty-bitmap-remove",
>> -      "arguments": {
>> -        "node": "drive0",
>> -        "name": "bitmap0"
>> -      }
>> -    }
>> +.. admonition:: Example
>>   
>> -Resetting
>> -~~~~~~~~~
>> + Remove a bitmap named ``bitmap0`` from node ``drive0``:
>>   
>> --  Resetting a bitmap will clear all information it holds.
>> + .. code:: json
>> +
>> +  -> { "execute": "block-dirty-bitmap-remove",
>> +       "arguments": {
>> +         "node": "drive0",
>> +         "name": "bitmap0"
>> +       }
>> +     }
>> +
>> +  <- { "return": {} }
>> +
>> +Resetting: block-dirty-bitmap-clear
>> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> +
>> +`block-dirty-bitmap-clear
>> +<qemu-qmp-ref.html#index-block_002ddirty_002dbitmap_002dclear>`_:
>> +
>> +Clears all dirty bits from a bitmap. ``+busy`` bitmaps cannot be cleared.
>>   
>>   -  An incremental backup created from an empty bitmap will copy no data,
>>      as if nothing has changed.
>>   
>> -.. code:: json
>> -
>> -    { "execute": "block-dirty-bitmap-clear",
>> -      "arguments": {
>> -        "node": "drive0",
>> -        "name": "bitmap0"
>> -      }
>> -    }
>> +.. admonition:: Example
>> +
>> + Clear all dirty bits from bitmap ``bitmap0`` on node ``drive0``:
>> +
>> + .. code:: json
>> +
>> +  -> { "execute": "block-dirty-bitmap-clear",
>> +       "arguments": {
>> +         "node": "drive0",
>> +         "name": "bitmap0"
>> +       }
>> +     }
>> +
>> +  <- { "return": {} }
>> +
>> +Enabling: block-dirty-bitmap-enable
>> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> +
>> +`block-dirty-bitmap-enable
>> +<qemu-qmp-ref.html#index-block_002ddirty_002dbitmap_002denable>`_:
>> +
>> +"Enables" a bitmap, setting the ``recording`` bit to true, causing guest 
>> writes
> 
> may be not guest, but any writes to the node
> 

[OK, I'll look for any time I mention guest writes and try to improve
the wording.]

>> +to begin being recorded. ``+busy`` bitmaps cannot be enabled.
> 
> hmm, you never mentions that +inconsistent also restrict most of opertaions
> 

It comes quite a bit later, where I suggest that the only valid
operation on an inconsistent bitmap is "remove," but I can try to pay it
some homage here, too.

>> +
>> +- Bitmaps default to being enabled when created, unless configured 
>> otherwise.
>> +
>> +- Persistent enabled bitmaps will remember their ``+recording`` status on 
>> load.
>> +
>> +.. admonition:: Example
>> +
>> + To set ``+recording`` on bitmap ``bitmap0`` on node ``drive0``:
>> +
>> + .. code:: json
>> +
>> +  -> { "execute": "block-dirty-bitmap-enable",
>> +       "arguments": {
>> +         "node": "drive0",
>> +         "name": "bitmap0"
>> +       }
>> +     }
>> +
>> +  <- { "return": {} }
>> +
>> +Enabling: block-dirty-bitmap-disable
>> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> +
>> +`block-dirty-bitmap-disable
>> +<qemu-qmp-ref.html#index-block_002ddirty_002dbitmap_002ddisable>`_:
>> +
>> +"Disables" a bitmap, setting the ``recording`` bit to false, causing further
>> +guest writes to begin being ignored. ``+busy`` bitmaps cannot be disabled.
> 
> same comments here
> 

Worried that there's going to be a lot of this.
(At least I was consistently misleading?)

>> +
>> +.. warning::
>> +
>> +  This is potentially dangerous: QEMU makes no effort to stop any guest 
>> writes
>> +  if there are disabled bitmaps on a drive, and will not mark any disabled
>> +  bitmaps as ``+inconsistent`` if any such writes do happen. Backups made 
>> from
>> +  such bitmaps will not be able to be used to reconstruct a full guest 
>> image.
>> +
>> +- Disabling a bitmap may be useful for examining which sectors of a disk 
>> changed
>> +  during a specific time period, or for explicit management of differential
>> +  backup windows.
>> +
>> +- Persistent disabled bitmaps will remember their ``-recording`` status on 
>> load.
>> +
>> +.. admonition:: Example
>> +
>> + To set ``-recording`` on bitmap ``bitmap0`` on node ``drive0``:
>> +
>> + .. code:: json
>> +
>> +  -> { "execute": "block-dirty-bitmap-disable",
>> +       "arguments": {
>> +         "node": "drive0",
>> +         "name": "bitmap0"
>> +       }
>> +     }
>> +
>> +  <- { "return": {} }
>> +
>> +Merging, Copying: block-dirty-bitmap-merge
>> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> +
>> +`block-dirty-bitmap-merge
>> +<qemu-qmp-ref.html#index-block_002ddirty_002dbitmap_002dmerge>`_:
>> +
>> +Merges one or more bitmaps into a target bitmap. For any segment that is 
>> dirty
>> +in any one source bitmap, the target bitmap will mark that segment dirty.
>> +
>> +- Merge takes one or more bitmaps as a source and copies them into a single
>> +  destination.
> 
> disagree with term "copies" here.. copy is not merge.
> 

Hm... yeah, I will change this phrasing.

>> +
>> +- Merge does not create the destination bitmap if it does not exist. A blank
>> +  bitmap can be created beforehand to achieve the same effect.
> 
> (hmm, interesting, may be we want to add such a feature)
> 

It might make things one command easier in general, but also prohibits
you from doing things like setting the persistence bit.

I'm fairly ambivalent about this one in particular, but... maybe.

>> +
>> +- The destination is not cleared prior to merge, so subsequent merge 
>> operations
>> +  will continue to cumulatively mark more segments as dirty.
>> +
>> +- If the merge operation should fail, the destination bitmap is guaranteed 
>> to be
>> +  unmodified. The operation may fail if the source or destination bitmaps 
>> are
>> +  busy, or have different granularities.
>> +
>> +- Bitmaps can only be merged on the same node. There is only one "node"
>> +  argument, so all bitmaps must be attached to that same node.
>> +
>> +- Copy can be achieved by merging from a single source to an empty 
>> destination.
>> +
>> +.. admonition:: Example
>> +
>> + Merge the data from ``bitmap0`` into the bitmap ``new_bitmap`` on node
>> + ``drive0``. If ``new_bitmap`` was empty prior to this command, this 
>> achieves a
>> + copy.
>> +
>> + .. code:: json
>> +
>> +  -> { "execute": "block-dirty-bitmap-merge",
>> +       "arguments": {
>> +         "node": "drive0",
>> +         "target": "new_bitmap",
>> +         "bitmaps: [ "bitmap0" ]
>> +       }
>> +     }
>> +
>> +  <- { "return": {} }
>> +
>> +Querying: query-block
>> +~~~~~~~~~~~~~~~~~~~~~
>> +
>> +`query-block
>> +<qemu-qmp-ref.html#index-query_002dblock>`_:
>> +
>> +Not strictly a bitmaps command, but will return information about any 
>> bitmaps
>> +attached to nodes serving as the root for guest devices.
> 
> not for all nodes?
> 

It doesn't, no. This needs some attention for 4.1 actually. If you look
at test 124, when I expanded that test to cover the new status bits,I
ran into difficulty querying the nodes that weren't attached to device
models.

I want to add a block-dirty-bitmap-query command that just simply and
straightforwardly returns all of the bitmaps, by node...

Also, it's odd that query-block lists them by device instead of by
*node*... I believe there is a query-named-block-nodes that perhaps I
could augment with Bitmap Info, but this might impact the query-block
mechanism...

Anyway, I'll be working on this soon.

>> +
>> +- The "inconsistent" bit will not appear when it is false, appearing only 
>> when
>> +  the value is true to indicate there is a problem.
>> +
>> +.. admonition:: Example
>> +
>> + Query the block sub-system of QEMU. The following json has trimmed 
>> irrelevant
>> + keys from the response to highlight only the bitmap-relevant portions of 
>> the
>> + API. This result highlights a bitmap ``bitmap0`` attached to the root node 
>> of
>> + device ``drive0``.
>> +
>> + .. code:: json
>> +
>> +  -> {
>> +       "execute": "query-block",
>> +       "arguments": {}
>> +     }
>> +
>> +  <- {
>> +       "return": [ {
>> +         "dirty-bitmaps": [ {
>> +           "status": "active",
>> +           "count": 0,
>> +           "busy": false,
>> +           "name": "bitmap0",
>> +           "persistent": false,
>> +           "recording": true,
>> +           "granularity": 65536
>> +         } ],
>> +         "device": "drive0",
>> +       } ]
>> +     }
>> +
>> +Bitmap Persistence
>> +------------------
>> +
>> +As outlined in `Supported Image Formats`_, QEMU can persist bitmaps to qcow2
>> +files. Demonstrated in `Creation: block-dirty-bitmap-add`_, passing
>> +``persistent: true`` to ``block-dirty-bitmap-add`` will persist that bitmap 
>> to
>> +disk.
>> +
>> +Persistent bitmaps will be automatically loaded into memory upon load, and 
>> will
>> +be written back to disk upon close. Their usage should be mostly 
>> transparent.
>> +
>> +However, if QEMU does not get a chance to close the file cleanly, the bitmap
>> +will be marked as ``+inconsistent`` and considered unsafe to use for any
>> +operation. At this point, the only valid operation on such bitmaps is
>> +``block-dirty-bitmap-remove``.
> 
> Not exactly. If we failed to save bitmap, than we will not clear IN_USE bit 
> from qcow2
> for this bitmap, and on _next_ load qemu will mark the bitmap as 
> ``+inconsistent``.
> 

Welllll. Right, it's actually already been marked, but that's an
implementation detail. From the point of view of the user, the
+inconsistent flag shows up on next boot.

I can clarify that it's on the *next* boot that this shows up; but
there's little room for QEMU to do any such marking as its busy
segfaulting or aborting :)

>> +
>> +Losing a bitmap in this way does not invalidate any existing bitmaps that 
>> have
>> +been made, but no further backups will be able to be issued for this chain.
> 
> And you said nothing about any chains before.. Oh, it sounds better when I 
> understand
> that s/existing bitmaps/existing backups/
> 

Ah, yes. Existing backups, not bitmaps.

>>   
>>   Transactions
>>   ------------
>>   
>> +Transactions are a QMP feature that allows you to submit multiple QMP 
>> commands
>> +at once, being guaranteed that they will all succeed or fail atomically,
>> +together. The interaction of bitmaps and transactions are demonstrated 
>> below.
>> +
>> +See `transaction <qemu-qmp.ref.html#index-transaction>`_ in the QMP 
>> reference
>> +for more details.
>> +
>>   Justification
>>   ~~~~~~~~~~~~~
>>   
>> -Bitmaps can be safely modified when the VM is paused or halted by using
>> -the basic QMP commands. For instance, you might perform the following
>> -actions:
>> +Bitmaps can generally be modified at any time, but certain operations often 
>> only
>> +make sense when paired directly with other commands. When a VM is paused, 
>> it's
>> +easy to ensure that no guest writes occur between individual QMP commands. 
>> When
>> +a VM is running, this is difficult to accomplish with individual QMP 
>> commands
>> +that may allow guest writes to occur inbetween each command.
>>   
>> -1. Boot the VM in a paused state.
>> -2. Create a full drive backup of drive0.
>> -3. Create a new bitmap attached to drive0.
>> -4. Resume execution of the VM.
>> -5. Incremental backups are ready to be created.
>> +For example, using only individual QMP commands, we could:
>> +
>> +#. Boot the VM in a paused state.
>> +#. Create a full drive backup of drive0.
>> +#. Create a new bitmap attached to drive0, confident that nothing has been
>> +   written to drive0 in the meantime.
>> +#. Resume execution of the VM.
>> +#. At a later point, issue incremental backups from ``bitmap0``.
>>   
>>   At this point, the bitmap and drive backup would be correctly in sync,
>>   and incremental backups made from this point forward would be correctly
>> @@ -140,416 +493,946 @@ This is not particularly useful if we decide we want 
>> to start
>>   incremental backups after the VM has been running for a while, for which
>>   we will need to perform actions such as the following:
>>   
>> -1. Boot the VM and begin execution.
>> -2. Using a single transaction, perform the following operations:
>> +#. Boot the VM and begin execution.
>> +#. Using a single transaction, perform the following operations:
>>   
>>      -  Create ``bitmap0``.
>>      -  Create a full drive backup of ``drive0``.
>>   
>> -3. Incremental backups are now ready to be created.
>> +#. At a later point, issue incremental backups from ``bitmap0``.
> 
> 
> Interesting will it work just without a transaction, create bitmap first, and 
> then full backup?
> If we'll have any intermediate writes, they will increase extra data in first 
> incremental backup
> bit it is not wrong. I only doubt about some unfinished requests.. Does 
> transaction give us
> drained point and do we really need it?
> 

Ah, you know... you're right. It will be just extra data and not
"wrong". I guess I wanted to encourage "good habits" with being careful
about the time slices.

...but you ARE right.

I wonder if it's worth clarifying this, or if that's just more confusing...

(By the way, I want to send a patch that allows you to specify a bitmap
with sync=full that clears the bitmap upon completion, as a convenience.
I think it's not very hard to add.)

>>   
>>   Supported Bitmap Transactions
>>   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>   
>>   -  ``block-dirty-bitmap-add``
>>   -  ``block-dirty-bitmap-clear``
>> +-  ``block-dirty-bitmap-enable``
>> +-  ``block-dirty-bitmap-disable``
>> +-  ``block-dirty-bitmap-merge``
>>   
>> -The usages are identical to their respective QMP commands, but see below
>> -for examples.
>> +The usages for these commands are identical to their respective QMP 
>> commands,
>> +but see the examples in the sections below for concrete examples.
> 
> see examples for examples
> 

LOL.

>>   
>> -Example: New Incremental Backup
>> -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> +Incremental Backups - Push Model
>> +--------------------------------
>>   
>> -As outlined in the justification, perhaps we want to create a new
>> -incremental backup chain attached to a drive.
>> +Incremental backups are simply partial disk images that can be combined with
>> +other partial disk images on top of a base image to reconstruct a full 
>> backup
>> +from the point in time at which the incremental backup was issued.
>>   
>> -.. code:: json
>> +The "Push Model" here references the fact that QEMU is "pushing" the 
>> modified
>> +blocks out to a destination. We will be using the `drive-backup
>> +<qemu-qmp-ref.html#index-drive_002dbackup>`_ and `blockdev-backup
>> +<qemu-qmp-ref.html#index-blockdev_002dbackup>`_ QMP commands to create both
>> +full and incremental backups.
>>   
>> -    { "execute": "transaction",
>> -      "arguments": {
>> -        "actions": [
>> -          {"type": "block-dirty-bitmap-add",
>> -           "data": {"node": "drive0", "name": "bitmap0"} },
>> -          {"type": "drive-backup",
>> -           "data": {"device": "drive0", "target": 
>> "/path/to/full_backup.img",
>> -                    "sync": "full", "format": "qcow2"} }
>> -        ]
>> -      }
>> -    }
>> +Both of these commands are jobs, which have their own QMP API for querying 
>> and
>> +management documented in `Background jobs
>> +<qemu-qmp-ref.html#Background-jobs>`_.
>>   
>>   Example: New Incremental Backup Anchor Point
>>   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>   
>> -Maybe we just want to create a new full backup with an existing bitmap
>> -and want to reset the bitmap to track the new chain.
>> +As outlined in the Transactions - `Justification`_ section, perhaps we want 
>> to
>> +create a new incremental backup chain attached to a drive.
>> +
>> +This example creates a new, full backup of "drive0" and accompanies it with 
>> a
>> +new, empty bitmap that records guest writes from this point in time forward.
>> +
>> +.. note:: Any new writes that happen after this command is issued, even 
>> while
>> +          the backup job runs, will be written locally and not to the backup
>> +          destination. These writes will be recorded in the bitmap 
>> accordingly.
>>   
>>   .. code:: json
>>   
>> -    { "execute": "transaction",
>> -      "arguments": {
>> -        "actions": [
>> -          {"type": "block-dirty-bitmap-clear",
>> -           "data": {"node": "drive0", "name": "bitmap0"} },
>> -          {"type": "drive-backup",
>> -           "data": {"device": "drive0", "target": 
>> "/path/to/new_full_backup.img",
>> -                    "sync": "full", "format": "qcow2"} }
>> -        ]
>> -      }
>> -    }
>> -
>> -Incremental Backups
>> --------------------
>> -
>> -The star of the show.
>> -
>> -**Nota Bene!** Only incremental backups of entire drives are supported
>> -for now. So despite the fact that you can attach a bitmap to any
>> -arbitrary node, they are only currently useful when attached to the root
>> -node. This is because drive-backup only supports drives/devices instead
>> -of arbitrary nodes.
>> +  -> {
>> +       "execute": "transaction",
>> +       "arguments": {
>> +         "actions": [
>> +           {
>> +             "type": "block-dirty-bitmap-add",
>> +             "data": {
>> +               "node": "drive0",
>> +               "name": "bitmap0"
>> +             }
>> +           },
>> +           {
>> +             "type": "drive-backup",
>> +             "data": {
>> +               "device": "drive0",
>> +               "target": "/path/to/full_backup.qcow2",
>> +               "sync": "full",
>> +               "format": "qcow2"
>> +             }
>> +           }
>> +         ]
>> +       }
>> +     }
>> +
>> +  <- { "return": {} }
>> +
>> +  <- {
>> +       "timestamp": {
>> +         "seconds": 1555436945,
>> +         "microseconds": 179620
>> +       },
>> +       "data": {
>> +         "status": "created",
>> +         "id": "drive0"
>> +       },
>> +       "event": "JOB_STATUS_CHANGE"
>> +     }
>> +
>> +  ...
>> +
>> +  <- {
>> +       "timestamp": {...},
>> +       "data": {
>> +         "device": "drive0",
>> +         "type": "backup",
>> +         "speed": 0,
>> +         "len": 68719476736,
>> +         "offset": 68719476736
>> +       },
>> +       "event": "BLOCK_JOB_COMPLETED"
>> +     }
>> +
>> +  <- {
>> +       "timestamp": {...},
>> +       "data": {
>> +         "status": "concluded",
>> +         "id": "drive0"
>> +       },
>> +       "event": "JOB_STATUS_CHANGE"
>> +     }
>> +
>> +  <- {
>> +       "timestamp": {...},
>> +       "data": {
>> +         "status": "null",
>> +         "id": "drive0"
>> +       },
>> +       "event": "JOB_STATUS_CHANGE"
>> +     }
>> +
>> +A full explanation of the job transition semantics and the JOB_STATUS_CHANGE
>> +event are beyond the scope of this document and will be omitted in all
>> +subsequent examples; above, several more events have been omitted for 
>> brevity.
>> +
>> +Events above have had their timestamp objects omitted for brevity.
> 
> ('"timestamp": {...}' is obvious notation, anyway, and a bit in conflict with 
> last
> sentence)
> 

So just omit this line in your opinion? The old document made no
omissions at all, so I was just trying to be honest about cutting stuff
out that was really not relevant to this presentation.

>> +
>> +.. note:: Subsequent examples will omit all events except 
>> BLOCK_JOB_COMPLETED
>> +          except where necessary to illustrate workflow differences.
>> +
>> +          Omitted events and json objects will be represented by ellipses:
>> +          ``...``
>> +
>> +Example: Resetting an Incremental Backup Anchor Point
>> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> +
>> +If we want to start a new backup chain with an existing bitmap, we can also 
>> use
>> +a transaction to reset the bitmap while making a new full backup:
>> +
>> +.. code:: json
>> +
>> +  -> {
>> +       "execute": "transaction",
>> +       "arguments": {
>> +         "actions": [
>> +         {
>> +           "type": "block-dirty-bitmap-clear",
>> +           "data": {
>> +             "node": "drive0",
>> +             "name": "bitmap0"
>> +           }
>> +         },
>> +         {
>> +           "type": "drive-backup",
>> +           "data": {
>> +             "device": "drive0",
>> +             "target": "/path/to/new_full_backup.qcow2",
>> +             "sync": "full",
>> +             "format": "qcow2"
>> +           }
>> +         }
>> +       ]
>> +     }
>> +   }
>> +
>> +  <- { "return": {} }
>> +
>> +  ...
>> +
>> +  <- {
>> +       "timestamp": {...},
>> +       "data": {
>> +         "device": "drive0",
>> +         "type": "backup",
>> +         "speed": 0,
>> +         "len": 68719476736,
>> +         "offset": 68719476736
>> +       },
>> +       "event": "BLOCK_JOB_COMPLETED"
>> +     }
>> +
>> +  ...
>> +
>> +The result of this example is identical to the first, but we clear an 
>> existing
>> +bitmap instead of adding a new one.
>> +
>> +.. tip:: In both of these examples, "bitmap0" is tied conceptually to the
>> +         creation of new, full backups. This relationship is not saved or
>> +         remembered by QEMU; it is up to the operator or management layer to
>> +         remember which bitmaps are associated with which backups.
>>   
>>   Example: First Incremental Backup
>>   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>   
>> -1. Create a full backup and sync it to the dirty bitmap, as in the
>> -   transactional examples above; or with the VM offline, manually create
>> -   a full copy and then create a new bitmap before the VM begins
>> -   execution.
>> +#. Create a full backup and sync it to the dirty bitmap using either of the 
>> two
>> +   example methods above, or, with the VM offline as suggested in the
>> +   Transactions `Justification`_ section, manually create a full copy and 
>> then
>> +   create a new bitmap before the VM begins execution.
> 
> Justification suggested not exactly this, but start in paused mode and do 
> full backup.
> So, actually manual copy when VM is offline, then start in pause mode and 
> create bitmaps,
> then start - is yet another method
> 

True...

>>   
>> -   -  Let's assume the full backup is named ``full_backup.img``.
>> -   -  Let's assume the bitmap you created is ``bitmap0`` attached to
>> +   - Let's assume the full backup is named ``full_backup.qcow2``.
>> +   - Let's assume the bitmap we created is named ``bitmap0``, attached to
>>         ``drive0``.
>>   
>> -2. Create a destination image for the incremental backup that utilizes
>> +#. Create a destination image for the incremental backup that utilizes
>>      the full backup as a backing image.
>>   
>>      -  Let's assume the new incremental image is named
>> -      ``incremental.0.img``.
>> +      ``incremental.0.qcow2``:
>>   
>>      .. code:: bash
>>   
>> -       $ qemu-img create -f qcow2 incremental.0.img -b full_backup.img -F 
>> qcow2
>> +       $ qemu-img create -f qcow2 incremental.0.qcow2 \
>> +         -b full_backup.qcow2 -F qcow2
>>   
>> -3. Issue the incremental backup command:
>> +#. Issue an incremental backup command:
>>   
>>      .. code:: json
>>   
>> -       { "execute": "drive-backup",
>> +    -> {
>> +         "execute": "drive-backup",
>>            "arguments": {
>>              "device": "drive0",
>>              "bitmap": "bitmap0",
>> -           "target": "incremental.0.img",
>> +           "target": "incremental.0.qcow2",
>>              "format": "qcow2",
>>              "sync": "incremental",
>>              "mode": "existing"
>>            }
>>          }
>>   
>> +    <- { "return": {} }
>> +
>> +    ...
>> +
>> +    <- {
>> +         "timestamp": {...},
>> +         "data": {
>> +           "device": "drive0",
>> +           "type": "backup",
>> +           "speed": 0,
>> +           "len": 68719476736,
>> +           "offset": 68719476736
>> +         },
>> +         "event": "BLOCK_JOB_COMPLETED"
>> +       }
>> +
>> +    ...
>> +
>> +This copies any blocks modified since the full backup was created into the
>> +``incremental.0.qcow2`` file. During the operation, ``bitmap0`` is marked
>> +``+busy``. If the operation is successful, ``bitmap0`` will be cleared to
>> +reflect the "incremental" backup regimen, which only copies out new changes
>> +from each incremental backup.
>> +
>> +.. note:: Any new writes that occur after the backup operation starts do not
>> +          get copied to the destination. The backup's "point in time" is 
>> when
>> +          the backup starts, not when it ends. These writes are recorded in 
>> a
>> +          special bitmap that gets re-added to bitmap0 when the backup ends 
>> so
>> +          that the next incremental backup can copy them out.
>> +
>>   Example: Second Incremental Backup
>>   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>   
>> -1. Create a new destination image for the incremental backup that points
>> -   to the previous one, e.g.: ``incremental.1.img``
>> +#. Create a new destination image for the incremental backup that points
>> +   to the previous one, e.g.: ``incremental.1.qcow2``
>>   
>>      .. code:: bash
>>   
>> -       $ qemu-img create -f qcow2 incremental.1.img -b incremental.0.img -F 
>> qcow2
>> +       $ qemu-img create -f qcow2 incremental.1.qcow2 \
>> +         -b incremental.0.qcow2 -F qcow2
>>   
>> -2. Issue a new incremental backup command. The only difference here is
>> +#. Issue a new incremental backup command. The only difference here is
>>      that we have changed the target image below.
>>   
>>      .. code:: json
>>   
>> -       { "execute": "drive-backup",
>> +    -> {
>> +         "execute": "drive-backup",
>>            "arguments": {
>>              "device": "drive0",
>>              "bitmap": "bitmap0",
>> -           "target": "incremental.1.img",
>> +           "target": "incremental.1.qcow2",
>>              "format": "qcow2",
>>              "sync": "incremental",
>>              "mode": "existing"
>>            }
>>          }
>>   
>> -Errors
>> -------
>> +    <- { "return": {} }
>>   
>> --  In the event of an error that occurs after a backup job is
>> -   successfully launched, either by a direct QMP command or a QMP
>> -   transaction, the user will receive a ``BLOCK_JOB_COMPLETE`` event with
>> -   a failure message, accompanied by a ``BLOCK_JOB_ERROR`` event.
>> +    ...
>> +
>> +    <- {
>> +         "timestamp": {...},
>> +         "data": {
>> +           "device": "drive0",
>> +           "type": "backup",
>> +           "speed": 0,
>> +           "len": 68719476736,
>> +           "offset": 68719476736
>> +         },
>> +         "event": "BLOCK_JOB_COMPLETED"
>> +       }
>> +
>> +    ...
>> +
>> +Because the first incremental backup from the previous example completed
>> +successfully, ``bitmap0`` was synchronized with ``incremental.0.qcow2``. 
>> Here,
>> +we use ``bitmap0`` again to create a new incremental backup that targets the
>> +previous one, creating a chain of three images:
>> +
>> +.. admonition:: Diagram
>> +
>> + .. code:: text
>> +
>> +   +--------+   +---------------+   +---------------+
>> +   |        |   |               |   |               |
>> +   |  base  +<--+ incremental.0 +<--+ incremental.1 |
>> +   |        |   |               |   |               |
>> +   +--------+   +---------------+   +---------------+
>> +
>> +Each new incremental backup re-synchronizes the bitmap to the latest backup
>> +authored, allowing a user to continue to "consume" it to create new backups
>> +on top of an existing chain.
>> +
>> +In the above diagram, incremental.1 represents incremental.1.qcow2; it is 
>> not a
>> +complete image by itself but relies on backing files to reconstruct a full
>> +image. incremental.0 similarly requires the full base image to reconstruct a
>> +functioning image.
>> +
>> +Each backup in this chain remains independent, and is unchanged by new 
>> entries
>> +made later in the chain. For instance, incremental.0 remains a perfectly 
>> valid
>> +backup of the disk as it was when the backup was issued.
>> +
>> +Example: Incremental Push Backups without Backing Files
>> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> +
>> +Backup images are best kept off-site, so we often will not have the 
>> preceding
>> +backups in a chain available to link against. This is not a problem at 
>> backup
>> +time; we simply do not set the backing image when creating the destination
>> +image:
>> +
>> +#. Create a new destination image with no backing file set. We will need to
>> +   specify the size of the base image this time, because it isn't available 
>> for
>> +   QEMU to use to guess:
> 
> or just drop "mode: existing"
> 

Usually true; but you lose control over the compatibility flags and such
which I am glossing over entirely here. We usually advocate for people
to make the destination images directly because the facilities to
auto-create the image on demand are not consistently great.

Eh, maybe I'm being overcautious and it's fine. It's how this document
*was* written in the past, so I didn't attempt to change it.

>> +
>> +   .. code:: bash
>> +
>> +       $ qemu-img create -f qcow2 incremental.2.qcow2 64G
>> +
>> +#. Issue a new incremental backup command. Apart from the new destination 
>> image,
>> +   there is no difference from the last two examples.
>> +
>> +   .. code:: json
>> +
>> +    -> {
>> +         "execute": "drive-backup",
>> +         "arguments": {
>> +           "device": "drive0",
>> +           "bitmap": "bitmap0",
>> +           "target": "incremental.2.qcow2",
>> +           "format": "qcow2",
>> +           "sync": "incremental",
>> +           "mode": "existing"
>> +         }
>> +       }
>> +
>> +    <- { "return": {} }
>> +
>> +    ...
>> +
>> +    <- {
>> +         "timestamp": {...},
>> +         "data": {
>> +           "device": "drive0",
>> +           "type": "backup",
>> +           "speed": 0,
>> +           "len": 68719476736,
>> +           "offset": 68719476736
>> +         },
>> +         "event": "BLOCK_JOB_COMPLETED"
>> +       }
>> +
>> +    ...
>> +
>> +The only difference from the perspective of the user is that you will need 
>> to
>> +set the backing image when attempting to restore the backup:
>> +
>> +.. code:: bash
>> +
>> +    $ qemu-img rebase incremental.2.qcow2 \
>> +      -u -b incremental.1.qcow2
>> +
>> +This uses the "unsafe" rebase mode to simply set the backing file to a file 
>> that
>> +isn't present.
> 
> or just instruct use --image-opts to specify the whole backing chain by hand 
> for
> qemu-img convert or qemu-nbd (which is used for restore)
> 

Ah, yeah. I can try to expand on that.

>> +
>> +Example: Multi-drive Incremental Backup
>> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> +
>> +Assume we have a VM with two drives, "drive0" and "drive1" and we wish to 
>> back
>> +both of them up such that the two backups represent the same 
>> crash-consistent
>> +point in time.
>> +
>> +#. For each drive, create an empty image:
>> +
>> +   .. code:: bash
>> +
>> +    $ qemu-img create -f qcow2 drive0.full.qcow2 64G
>> +    $ qemu-img create -f qcow2 drive1.full.qcow2 64G
>> +
>> +#. Create a full (anchor) backup for each drive, with accompanying bitmaps:
>> +
>> +   .. code:: json
>> +
>> +    -> {
>> +         "execute": "transaction",
>> +         "arguments": {
>> +           "actions": [
>> +             {
>> +               "type": "block-dirty-bitmap-add",
>> +               "data": {
>> +                 "node": "drive0",
>> +                 "name": "bitmap0"
>> +               }
>> +             },
>> +             {
>> +               "type": "block-dirty-bitmap-add",
>> +               "data": {
>> +                 "node": "drive1",
>> +                 "name": "bitmap0"
>> +               }
>> +             },
>> +             {
>> +               "type": "drive-backup",
>> +               "data": {
>> +                 "device": "drive0",
>> +                 "target": "/path/to/drive0.full.qcow2",
>> +                 "sync": "full",
>> +                 "format": "qcow2"
>> +               }
>> +             },
>> +             {
>> +               "type": "drive-backup",
>> +               "data": {
>> +                 "device": "drive1",
>> +                 "target": "/path/to/drive1.full.qcow2",
>> +                 "sync": "full",
>> +                 "format": "qcow2"
>> +               }
>> +             }
>> +           ]
>> +         }
>> +       }
>> +
>> +    <- { "return": {} }
>> +
>> +    ...
>> +
>> +    <- {
>> +         "timestamp": {...},
>> +         "data": {
>> +           "device": "drive0",
>> +           "type": "backup",
>> +           "speed": 0,
>> +           "len": 68719476736,
>> +           "offset": 68719476736
>> +         },
>> +         "event": "BLOCK_JOB_COMPLETED"
>> +       }
>> +
>> +    ...
>> +
>> +    <- {
>> +         "timestamp": {...},
>> +         "data": {
>> +           "device": "drive1",
>> +           "type": "backup",
>> +           "speed": 0,
>> +           "len": 68719476736,
>> +           "offset": 68719476736
>> +         },
>> +         "event": "BLOCK_JOB_COMPLETED"
>> +       }
>> +
>> +    ...
>> +
>> +#. Later, create new destination images for each of the incremental backups
>> +   that point to their respective full backups:
>> +
>> +   .. code:: bash
>> +
>> +     $ qemu-img create -f qcow2 drive0.inc0.qcow2 \
>> +       -b drive0.full.qcow2 -F qcow2
>> +     $ qemu-img create -f qcow2 drive1.inc0.qcow2 \
>> +       -b drive1.full.qcow2 -F qcow2
>> +
>> +#. Issue a multi-drive incremental push backup transaction:
>> +
>> +   .. code:: json
>> +
>> +    -> {
>> +         "execute": "transaction",
>> +         "arguments": {
>> +           "actions": [
>> +             {
>> +               "type": "drive-backup",
>> +               "data": {
>> +                 "device": "drive0",
>> +                 "bitmap": "bitmap0",
>> +                 "format": "qcow2",
>> +                 "mode": "existing",
>> +                 "sync": "incremental",
>> +                 "target": "drive0.inc0.qcow2"
>> +               }
>> +             },
>> +             {
>> +               "type": "drive-backup",
>> +               "data": {
>> +                 "device": "drive1",
>> +                 "bitmap": "bitmap0",
>> +                 "format": "qcow2",
>> +                 "mode": "existing",
>> +                 "sync": "incremental",
>> +                 "target": "drive1.inc0.qcow2"
>> +               }
>> +             },
>> +           ]
>> +         }
>> +       }
>> +
>> +    <- { "return": {} }
>> +
>> +    ...
>> +
>> +    <- {
>> +         "timestamp": {...},
>> +         "data": {
>> +           "device": "drive0",
>> +           "type": "backup",
>> +           "speed": 0,
>> +           "len": 68719476736,
>> +           "offset": 68719476736
>> +         },
>> +         "event": "BLOCK_JOB_COMPLETED"
>> +       }
>> +
>> +    ...
>> +
>> +    <- {
>> +         "timestamp": {...},
>> +         "data": {
>> +           "device": "drive1",
>> +           "type": "backup",
>> +           "speed": 0,
>> +           "len": 68719476736,
>> +           "offset": 68719476736
>> +         },
>> +         "event": "BLOCK_JOB_COMPLETED"
>> +       }
>> +
>> +    ...
>> +
>> +Push Backup Errors & Recovery
>> +-----------------------------
>> +
>> +-  In the event of an error that occurs after a push backup job is 
>> successfully
>> +   launched, either by a direct QMP command or a QMP transaction, the user 
>> will
>> +   receive a ``BLOCK_JOB_COMPLETE`` event with a failure message, 
>> accompanied by
>> +   a ``BLOCK_JOB_ERROR`` event.
>>   
>>   -  In the case of an event being cancelled, the user will receive a
> 
> s/event/job
> 

Whoops!

>> -   ``BLOCK_JOB_CANCELLED`` event instead of a pair of COMPLETE and ERROR
>> -   events.
>> +   ``BLOCK_JOB_CANCELLED`` event instead of a pair of COMPLETE and ERROR 
>> events.
>>   
>> --  In either case, the incremental backup data contained within the
>> -   bitmap is safely rolled back, and the data within the bitmap is not
>> -   lost. The image file created for the failed attempt can be safely
>> -   deleted.
>> +-  In either failure case, the bitmap used for the failed operation is not
>> +   cleared. It will contain all of the dirty bits it did at the start of the
>> +   operation, plus any new bits that got marked during the operation.
>>   
>> --  Once the underlying problem is fixed (e.g. more storage space is
>> -   freed up), you can simply retry the incremental backup command with
>> -   the same bitmap.
>> +-  Effectively, the "point in time" that a bitmap is recording differences
>> +   against is rolled back to the issuance of the last successful incremental
>> +   backup, instead of being moved forward to the start of this now-failed
>> +   backup.
> 
> rolled back instead of being moved forward sounds strange.
> 
> If we are not moving forward then nothing to roll back.
> 
> For me it would be clearer something like this:
> 
> Effectively, everything works like there was no failed backup at all and you 
> can
> safely retry the operation using same dirty bitmap (except you have to remove 
> target
> image).
> 

You're right, "rolled back" is not the right term here. I will rethink
this explanation to avoid those terms.

> 
> 
> hmm, noticed just now, that you stopped using ".. admonition::" sections for 
> examples...
> also, it would simpler to review, if style changes and paragraph reflow go in 
> separate
> patch..
> 

Yeah, sorry! I used the example boxes in the section with QMP usage, but
used only code boxes in these sections where the entire purpose of each
section is an example broken up into parts.

If you create an "Example: " box here, and then put the list inside of
it, and then code boxes inside of THAT, it looks much worse.

It's going to be a little difficult to split style refactoring from this
changeset now, so unfortunately I will ask that we consider both for now.

>>   
>> -Example
>> -~~~~~~~
>> +-  Once the underlying problem is addressed (e.g. more storage space is
>> +   allocated on the destination), the incremental backup command can be 
>> retried
>> +   with the same bitmap.
>>   
>> -1. Create a target image:
>> +Example: Individual Failures
>> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> +
>> +Backup jobs that fail individually behave simply as described above. This
>> +example shows the simplest case:
>> +
>> +#. Create a target image:
>>   
>>      .. code:: bash
>>   
>> -       $ qemu-img create -f qcow2 incremental.0.img -b full_backup.img -F 
>> qcow2
>> +       $ qemu-img create -f qcow2 incremental.0.qcow \
>> +         -b full_backup.qcow -F qcow2
>>   
>> -2. Attempt to create an incremental backup via QMP:
>> +#. Attempt to create an incremental backup via QMP:
>>   
>>      .. code:: json
>>   
>> -       { "execute": "drive-backup",
>> +    -> {
>> +         "execute": "drive-backup",
>>            "arguments": {
>>              "device": "drive0",
>>              "bitmap": "bitmap0",
>> -           "target": "incremental.0.img",
>> +           "target": "incremental.0.qcow2",
>>              "format": "qcow2",
>>              "sync": "incremental",
>>              "mode": "existing"
>>            }
>>          }
>>   
>> -3. Receive an event notifying us of failure:
>> +    <- { "return": {} }
>> +
>> +   Note that the job is successfully accepted.
>> +
>> +3. Receive an event indicating failure:
> 
> s/3/# , and later several times
> 

Ah, good spot.

>>   
>>      .. code:: json
>>   
>> -       { "timestamp": { "seconds": 1424709442, "microseconds": 844524 },
>> -         "data": { "speed": 0, "offset": 0, "len": 67108864,
>> -                   "error": "No space left on device",
>> -                   "device": "drive1", "type": "backup" },
>> -         "event": "BLOCK_JOB_COMPLETED" }
>> +    <- {
>> +         "timestamp": {...},
>> +         "data": {
>> +           "speed": 0,
>> +           "offset": 0,
>> +           "len": 67108864,
>> +           "error": "No space left on device",
>> +           "device": "drive0",
>> +           "type": "backup"
>> +         },
>> +         "event": "BLOCK_JOB_COMPLETED"
>> +       }
> 
> Should not BLOCK_JOB_ERROR go first?
> 

I'll check. I didn't update the ordering here from what I wrote back then.

>>   
>> -4. Delete the failed incremental, and re-create the image.
>> +4. Delete the failed image, and re-create it.
>>   
>>      .. code:: bash
>>   
>> -       $ rm incremental.0.img
>> -       $ qemu-img create -f qcow2 incremental.0.img -b full_backup.img -F 
>> qcow2
>> +       $ rm incremental.0.qcow
> 
> s/qcow/qcow2
> 

Ah dang. I even looked for this typo specifically because I made it so
often.

>> +       $ qemu-img create -f qcow2 incremental.0.qcow2 \
>> +         -b full_backup.qcow2 -F qcow2
>>   
>>   5. Retry the command after fixing the underlying problem, such as
>>      freeing up space on the backup volume:
>>   
>>      .. code:: json
>>   
>> -       { "execute": "drive-backup",
>> +    -> {
>> +         "execute": "drive-backup",
>>            "arguments": {
>>              "device": "drive0",
>>              "bitmap": "bitmap0",
>> -           "target": "incremental.0.img",
>> +           "target": "incremental.0.qcow2",
>>              "format": "qcow2",
>>              "sync": "incremental",
>>              "mode": "existing"
>>            }
>>          }
>>   
>> +    <- { "return": {} }
>> +
>>   6. Receive confirmation that the job completed successfully:
>>   
>>      .. code:: json
>>   
>> -       { "timestamp": { "seconds": 1424709668, "microseconds": 526525 },
>> -         "data": { "device": "drive1", "type": "backup",
>> -                   "speed": 0, "len": 67108864, "offset": 67108864},
>> -         "event": "BLOCK_JOB_COMPLETED" }
>> +    <- {
>> +         "timestamp": {...},
>> +         "data": {
>> +           "device": "drive0",
>> +           "type": "backup",
>> +           "speed": 0,
>> +           "len": 67108864,
>> +           "offset": 67108864
>> +         },
>> +         "event": "BLOCK_JOB_COMPLETED"
>> +       }
>>   
>> -Partial Transactional Failures
>> -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> +Example: Partial Transactional Failures
>> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>   
>> --  Sometimes, a transaction will succeed in launching and return
>> -   success, but then later the backup jobs themselves may fail. It is
>> -   possible that a management application may have to deal with a
>> -   partial backup failure after a successful transaction.
>> +QMP commands like `query-block <qemu-qmp-ref.html#index-query_002dblock>`_
> 
> query-block ?
> 

Ah, what the heck did I do here? I think this was a search-and-replace
gone awry somehow. Sorry.

>> +conceptually only start a job, and so these transactions may succeed even if
>> +the job later fails. This might have surprising interactions with notions of
>> +how a "transaction" ought to behave.
>>   
>> --  If multiple backup jobs are specified in a single transaction, when
>> -   one of them fails, it will not interact with the other backup jobs in
>> -   any way.
>> +This distinction means that on occasion, a transaction containing such job
>> +launching commands may appear to succeed and return success, but later
>> +individual jobs associated with the transaction may fail. It is possible 
>> that a
>> +management application may have to deal with a partial backup failure after 
>> a
>> +"successful" transaction.
>>   
>> --  The job(s) that succeeded will clear the dirty bitmap associated with
>> -   the operation, but the job(s) that failed will not. It is not "safe"
>> -   to delete any incremental backups that were created successfully in
>> -   this scenario, even though others failed.
>> +If multiple backup jobs are specified in a single transaction, if one of 
>> those
>> +jobs fails, it will not interact with the other backup jobs in any way by
>> +default. The job(s) that succeeded will clear the dirty bitmap associated 
>> with
>> +the operation, but the job(s) that failed will not. It is therefore not 
>> safe to
>> +delete any incremental backups that were created successfully in this 
>> scenario,
>> +even though others failed.
>>   
>> -Example
>> -^^^^^^^
>> +This example illustrates a transaction with two backup jobs, where one fails
>> +and one succeeds:
>>   
>> --  QMP example highlighting two backup jobs:
>> +#. Issue the transaction to start a backup of both drives. Note that the
>> +   transaction is accepted, indicating that the jobs are started 
>> succesfully.
>>   
>>      .. code:: json
>>   
>> -       { "execute": "transaction",
>> +    -> {
>> +         "execute": "transaction",
>>            "arguments": {
>>              "actions": [
>> -             { "type": "drive-backup",
>> -               "data": { "device": "drive0", "bitmap": "bitmap0",
>> -                         "format": "qcow2", "mode": "existing",
>> -                         "sync": "incremental", "target": "d0-incr-1.qcow2" 
>> } },
>> -             { "type": "drive-backup",
>> -               "data": { "device": "drive1", "bitmap": "bitmap1",
>> -                         "format": "qcow2", "mode": "existing",
>> -                         "sync": "incremental", "target": "d1-incr-1.qcow2" 
>> } },
>> -           ]
>> +           {
>> +             "type": "drive-backup",
>> +             "data": {
>> +               "device": "drive0",
>> +               "bitmap": "bitmap0",
>> +               "format": "qcow2",
>> +               "mode": "existing",
>> +               "sync": "incremental",
>> +               "target": "drive0.inc0.qcow2"
>> +             }
>> +           },
>> +           {
>> +             "type": "drive-backup",
>> +             "data": {
>> +               "device": "drive1",
>> +               "bitmap": "bitmap0",
>> +               "format": "qcow2",
>> +               "mode": "existing",
>> +               "sync": "incremental",
>> +               "target": "drive1.inc0.qcow2"
>> +             }
>> +           }]
>>            }
>>          }
>>   
>> --  QMP example response, highlighting one success and one failure:
>> +    <- { "return": {} }
>>   
>> -   -  Acknowledgement that the Transaction was accepted and jobs were
>> -      launched:
>> +#. Receive notice that the first job has completed:
>>   
>> -      .. code:: json
>> +   .. code:: json
>> +
>> +    <- {
>> +         "timestamp": {...},
>> +         "data": {
>> +           "device": "drive0",
>> +           "type": "backup",
>> +           "speed": 0,
>> +           "len": 67108864,
>> +           "offset": 67108864
>> +         },
>> +         "event": "BLOCK_JOB_COMPLETED"
>> +       }
>>   
>> -          { "return": {} }
>> +#. Receive notice that the second job has failed:
>>   
>> -   -  Later, QEMU sends notice that the first job was completed:
>> +   .. code:: json
>>   
>> -      .. code:: json
>> +    <- {
>> +         "timestamp": {...},
>> +         "data": {
>> +           "device": "drive1",
>> +           "action": "report",
>> +           "operation": "read"
>> +         },
>> +         "event": "BLOCK_JOB_ERROR"
>> +       }
>>   
>> -          { "timestamp": { "seconds": 1447192343, "microseconds": 615698 },
>> -            "data": { "device": "drive0", "type": "backup",
>> -                       "speed": 0, "len": 67108864, "offset": 67108864 },
>> -            "event": "BLOCK_JOB_COMPLETED"
>> -          }
>> +    ...
>>   
>> -   -  Later yet, QEMU sends notice that the second job has failed:
>> +    <- {
>> +         "timestamp": {...},
>> +         "data": {
>> +           "speed": 0,
>> +           "offset": 0,
>> +           "len": 67108864,
>> +           "error": "Input/output error",
>> +           "device": "drive1",
>> +           "type": "backup"
>> +         },
>> +         "event": "BLOCK_JOB_COMPLETED"
>> +       }
>>   
>> -      .. code:: json
>> +At the conclusion of the above example, ``drive0.inc0.qcow2`` is valid and 
>> must
>> +be kept, but ``drive1.inc0.qcow2`` is incomplete and should be deleted. If a
>> +VM-wide incremental backup of all drives at a point-in-time is to be made, 
>> new
>> +backups for both drives will need to be made, taking into account that a new
>> +incremental backup for drive0 needs to be based on top of 
>> ``drive0.inc0.qcow2``.
>>   
>> -          { "timestamp": { "seconds": 1447192399, "microseconds": 683015 },
>> -            "data": { "device": "drive1", "action": "report",
>> -                      "operation": "read" },
>> -            "event": "BLOCK_JOB_ERROR" }
>> +In other words, at the conclusion of the above example, we'd have made only 
>> an
>> +incremental backup for drive0 but not drive1. The last VM-wide crash
>> +consistent backup we have access to in this case is the anchor point.
>>   
>> -      .. code:: json
>> +.. code:: text
>>   
>> -          { "timestamp": { "seconds": 1447192399, "microseconds":
>> -          685853 }, "data": { "speed": 0, "offset": 0, "len": 67108864,
>> -          "error": "Input/output error", "device": "drive1", "type":
>> -          "backup" }, "event": "BLOCK_JOB_COMPLETED" }
>> +          [drive0.full.qcow2] <-- [drive0.inc0.qcow2]
>> +          [drive1.full.qcow2]
>>   
>> --  In the above example, ``d0-incr-1.qcow2`` is valid and must be kept,
>> -   but ``d1-incr-1.qcow2`` is invalid and should be deleted. If a VM-wide
>> -   incremental backup of all drives at a point-in-time is to be made,
>> -   new backups for both drives will need to be made, taking into account
>> -   that a new incremental backup for drive0 needs to be based on top of
>> -   ``d0-incr-1.qcow2``.
>> +To repair this, issue a new incremental backup across both drives. The 
>> result
>> +will be backup chains that resemble the following:
>>   
>> -Grouped Completion Mode
>> -~~~~~~~~~~~~~~~~~~~~~~~
>> +.. code:: text
>>   
>> --  While jobs launched by transactions normally complete or fail on
>> -   their own, it is possible to instruct them to complete or fail
>> -   together as a group.
>> +          [drive0.full.qcow2] <-- [drive0.inc0.qcow2] <-- 
>> [drive0.inc1.qcow2]
>> +          [drive1.full.qcow2] <-------------------------- 
>> [drive1.inc1.qcow2]
>>   
>> --  QMP transactions take an optional properties structure that can
>> -   affect the semantics of the transaction.
>> +Example: Grouped Completion Mode
>> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>   
>> --  The "completion-mode" transaction property can be either "individual"
>> -   which is the default, legacy behavior described above, or "grouped,"
>> -   a new behavior detailed below.
>> +While jobs launched by transactions normally complete or fail individually,
>> +it's possible to instruct them to complete or fail together as a group. QMP
>> +transactions take an optional properties structure that can affect the
>> +behavior of the transaction.
>>   
>> --  Delayed Completion: In grouped completion mode, no jobs will report
>> -   success until all jobs are ready to report success.
>> +The ``completion-mode`` transaction property can be either ``individual`` 
>> which
>> +is the default legacy behavior described above, or ``grouped``, detailed 
>> below.
>>   
>> --  Grouped failure: If any job fails in grouped completion mode, all
>> -   remaining jobs will be cancelled. Any incremental backups will
>> -   restore their dirty bitmap objects as if no backup command was ever
>> -   issued.
>> +In ``grouped`` completion mode, no jobs will report success until all jobs 
>> are
>> +ready to report success. If any job fails, all other jobs will be canceled.
>>   
>> -   -  Regardless of if QEMU reports a particular incremental backup job
>> -      as CANCELLED or as an ERROR, the in-memory bitmap will be
>> -      restored.
>> +Regardless of if a participating incremental backup job failed or was 
>> canceled,
>> +their associated bitmaps will all be rolled back as in individual failure
>> +cases.
>>   
>> -Example
>> -^^^^^^^
>> +Here's the same multi-drive backup scenario from `Example: Partial
>> +Transactional Failures`_, but with the ``grouped`` completion-mode property
>> +applied:
>>   
>> --  Here's the same example scenario from above with the new property:
>> +#. Issue the multi-drive incremental backup transaction:
>>   
>>      .. code:: json
>>   
>> -       { "execute": "transaction",
>> +    -> {
>> +         "execute": "transaction",
>>            "arguments": {
>> -           "actions": [
>> -             { "type": "drive-backup",
>> -               "data": { "device": "drive0", "bitmap": "bitmap0",
>> -                         "format": "qcow2", "mode": "existing",
>> -                         "sync": "incremental", "target": "d0-incr-1.qcow2" 
>> } },
>> -             { "type": "drive-backup",
>> -               "data": { "device": "drive1", "bitmap": "bitmap1",
>> -                         "format": "qcow2", "mode": "existing",
>> -                         "sync": "incremental", "target": "d1-incr-1.qcow2" 
>> } },
>> -           ],
>>              "properties": {
>>                "completion-mode": "grouped"
>> -           }
>> +           },
>> +           "actions": [
>> +           {
>> +             "type": "drive-backup",
>> +             "data": {
>> +               "device": "drive0",
>> +               "bitmap": "bitmap0",
>> +               "format": "qcow2",
>> +               "mode": "existing",
>> +               "sync": "incremental",
>> +               "target": "drive0.inc0.qcow2"
>> +             }
>> +           },
>> +           {
>> +             "type": "drive-backup",
>> +             "data": {
>> +               "device": "drive1",
>> +               "bitmap": "bitmap0",
>> +               "format": "qcow2",
>> +               "mode": "existing",
>> +               "sync": "incremental",
>> +               "target": "drive1.inc0.qcow2"
>> +             }
>> +           }]
>>            }
>>          }
>>   
>> --  QMP example response, highlighting a failure for ``drive2``:
>> -
>> -   -  Acknowledgement that the Transaction was accepted and jobs were
>> -      launched:
>> -
>> -      .. code:: json
>> -
>> -          { "return": {} }
>> -
>> -   -  Later, QEMU sends notice that the second job has errored out, but
>> -      that the first job was also cancelled:
>> -
>> -      .. code:: json
>> -
>> -          { "timestamp": { "seconds": 1447193702, "microseconds": 632377 },
>> -            "data": { "device": "drive1", "action": "report",
>> -                      "operation": "read" },
>> -            "event": "BLOCK_JOB_ERROR" }
>> -
>> -      .. code:: json
>> -
>> -          { "timestamp": { "seconds": 1447193702, "microseconds": 640074 },
>> -            "data": { "speed": 0, "offset": 0, "len": 67108864,
>> -                      "error": "Input/output error",
>> -                      "device": "drive1", "type": "backup" },
>> -            "event": "BLOCK_JOB_COMPLETED" }
>> -
>> -      .. code:: json
>> -
>> -          { "timestamp": { "seconds": 1447193702, "microseconds": 640163 },
>> -            "data": { "device": "drive0", "type": "backup", "speed": 0,
>> -                      "len": 67108864, "offset": 16777216 },
>> -            "event": "BLOCK_JOB_CANCELLED" }
>> +#. Receive acknowledgement that the Transaction was accepted, and jobs were
>> +   launched:
>> +
>> +    <- { "return": {} }
> 
> in previous example, you instead add a note:
>    Note that the
>    transaction is accepted, indicating that the jobs are started succesfully.
> 

Will make consistent. Thank you for this attention to detail.

>> +
>> +#. Receive notification that the backup job for ``drive1`` has failed:
>> +
>> +   .. code:: json
>> +
>> +    <- {
>> +         "timestamp": {...},
>> +         "data": {
>> +           "device": "drive1",
>> +           "action": "report",
>> +           "operation": "read"
>> +         },
>> +         "event": "BLOCK_JOB_ERROR"
>> +       }
>> +
>> +    <- {
>> +         "timestamp": {...},
>> +         "data": {
>> +           "speed": 0,
>> +           "offset": 0,
>> +           "len": 67108864,
>> +           "error": "Input/output error",
>> +           "device": "drive1",
>> +           "type": "backup"
>> +         },
>> +         "event": "BLOCK_JOB_COMPLETED"
>> +       }
>> +
>> +#. Receive notification that the job for ``drive0`` has been canceled:
>> +
>> +   .. code:: json
>> +
>> +    <- {
>> +         "timestamp": {...}
>> +         "data": {
>> +           "device": "drive0",
>> +           "type": "backup",
>> +           "speed": 0,
>> +           "len": 67108864,
>> +           "offset": 16777216
>> +         },
>> +         "event": "BLOCK_JOB_CANCELLED"
>> +       }
>>   
> 
> Good to add here some conclusion for example, like removing failed target 
> images
> and note the the transaction operation may be safely retried.
> 

Agree.

> 
>>   .. raw:: html
>>   
>>      <!--
>>      The FreeBSD Documentation License
>>   
>> -   Redistribution and use in source (Markdown) and 'compiled' forms (SGML, 
>> HTML,
>> -   PDF, PostScript, RTF and so forth) with or without modification, are 
>> permitted
>> -   provided that the following conditions are met:
>> +   Redistribution and use in source (ReST) and 'compiled' forms (SGML, HTML,
>> +   PDF, PostScript, RTF and so forth) with or without modification, are
>> +   permitted provided that the following conditions are met:
>>   
>> -   Redistributions of source code (Markdown) must retain the above copyright
>> -   notice, this list of conditions and the following disclaimer of this file
>> -   unmodified.
>> +   Redistributions of source code (ReST) must retain the above copyright 
>> notice,
>> +   this list of conditions and the following disclaimer of this file 
>> unmodified.
>>   
>> -   Redistributions in compiled form (transformed to other DTDs, converted 
>> to PDF,
>> -   PostScript, RTF and other formats) must reproduce the above copyright 
>> notice,
>> -   this list of conditions and the following disclaimer in the 
>> documentation and/or
>> -   other materials provided with the distribution.
>> +   Redistributions in compiled form (transformed to other DTDs, converted to
>> +   PDF, PostScript, RTF and other formats) must reproduce the above 
>> copyright
>> +   notice, this list of conditions and the following disclaimer in the
>> +   documentation and/or other materials provided with the distribution.
>>   
>> -   THIS DOCUMENTATION IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 
>> "AS IS"
>> -   AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
>> -   IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR  
>> PURPOSE ARE
>> -   DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS  BE 
>> LIABLE
>> -   FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR 
>> CONSEQUENTIAL
>> -   DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 
>> OR
>> -   SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 
>> HOWEVER
>> -   CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 
>> LIABILITY,
>> -   OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF 
>> THE USE OF
>> -   THIS DOCUMENTATION, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
>> +   THIS DOCUMENTATION IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 
>> "AS
>> +   IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED 
>> TO, THE
>> +   IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR 
>> PURPOSE
>> +   ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
>> +   LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
>> +   CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
>> +   SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
>> +   INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
>> +   CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
>> +   ARISING IN ANY WAY OUT OF THE USE OF THIS DOCUMENTATION, EVEN IF ADVISED 
>> OF
>> +   THE POSSIBILITY OF SUCH DAMAGE.
>>      -->
>> diff --git a/Makefile b/Makefile
>> index 04a0d45050..ff9ce2ed4c 100644
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -899,7 +899,7 @@ docs/version.texi: $(SRC_PATH)/VERSION
>>   sphinxdocs: $(MANUAL_BUILDDIR)/devel/index.html 
>> $(MANUAL_BUILDDIR)/interop/index.html
>>   
>>   # Canned command to build a single manual
>> -build-manual = $(call quiet-command,sphinx-build $(if $(V),,-q) -b html -D 
>> version=$(VERSION) -D release="$(FULL_VERSION)" -d .doctrees/$1 
>> $(SRC_PATH)/docs/$1 $(MANUAL_BUILDDIR)/$1 ,"SPHINX","$(MANUAL_BUILDDIR)/$1")
>> +build-manual = $(call quiet-command,sphinx-build $(if $(V),,-q) -n -b html 
>> -D version=$(VERSION) -D release="$(FULL_VERSION)" -d .doctrees/$1 
>> $(SRC_PATH)/docs/$1 $(MANUAL_BUILDDIR)/$1 ,"SPHINX","$(MANUAL_BUILDDIR)/$1")
> 
> what is '-n'?
> 

It complains about missing anchor references. I left it in by accident,
but I think we ought to check it in separately.

>>   # We assume all RST files in the manual's directory are used in it
>>   manual-deps = $(wildcard $(SRC_PATH)/docs/$1/*.rst) 
>> $(SRC_PATH)/docs/$1/conf.py $(SRC_PATH)/docs/conf.py
>>   
>>
> 
> 

Thank you for the review! I will address what I can before adding new
sections for differential and push backups. I'd like to check this in to
revise our existing docs before moving on to add new stuff.

--js



reply via email to

[Prev in Thread] Current Thread [Next in Thread]