diff options
| author | Ævar Arnfjörð Bjarmason <avarab@gmail.com> | 2022-08-04 18:28:36 +0200 |
|---|---|---|
| committer | Junio C Hamano <gitster@pobox.com> | 2022-08-04 14:12:23 -0700 |
| commit | 5db921054e685a4dbaeb622acda53d6a154e947f (patch) | |
| tree | d9dd16cb9c9cd7dd97747599b8417d07a010a559 /Documentation/technical | |
| parent | 8cbace93d270dc49f007a2c1922769240a22eca5 (diff) | |
| download | git-5db921054e685a4dbaeb622acda53d6a154e947f.tar.gz | |
docs: move protocol-related docs to man section 5
Continue the move of existing Documentation/technical/* protocol and
file-format documentation into our main documentation space. By moving
the things that discuss the protocol we can properly link from
e.g. lsrefs.unborn and protocol.version documentation to a manpage we
build by default.
So far we have been using the "gitformat-" prefix for the
documentation we've been moving over from Documentation/technical/*,
but for protocol documentation let's use "gitprotocol-*".
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Diffstat (limited to 'Documentation/technical')
| -rw-r--r-- | Documentation/technical/api-simple-ipc.txt | 2 | ||||
| -rw-r--r-- | Documentation/technical/http-protocol.txt | 6 | ||||
| -rw-r--r-- | Documentation/technical/long-running-process-protocol.txt | 2 | ||||
| -rw-r--r-- | Documentation/technical/pack-protocol.txt | 709 | ||||
| -rw-r--r-- | Documentation/technical/packfile-uri.txt | 2 | ||||
| -rw-r--r-- | Documentation/technical/partial-clone.txt | 2 | ||||
| -rw-r--r-- | Documentation/technical/protocol-capabilities.txt | 380 | ||||
| -rw-r--r-- | Documentation/technical/protocol-common.txt | 99 | ||||
| -rw-r--r-- | Documentation/technical/protocol-v2.txt | 568 |
9 files changed, 7 insertions, 1763 deletions
diff --git a/Documentation/technical/api-simple-ipc.txt b/Documentation/technical/api-simple-ipc.txt index d79ad323e6..d44ada98e7 100644 --- a/Documentation/technical/api-simple-ipc.txt +++ b/Documentation/technical/api-simple-ipc.txt @@ -78,7 +78,7 @@ client and an optional response message from the server. Both the client and server messages are unlimited in length and are terminated with a flush packet. -The pkt-line routines (Documentation/technical/protocol-common.txt) +The pkt-line routines (linkgit:gitprotocol-common[5]) are used to simplify buffer management during message generation, transmission, and reception. A flush packet is used to mark the end of the message. This allows the sender to incrementally generate and diff --git a/Documentation/technical/http-protocol.txt b/Documentation/technical/http-protocol.txt index cc5126cfed..8bd672d55b 100644 --- a/Documentation/technical/http-protocol.txt +++ b/Documentation/technical/http-protocol.txt @@ -222,7 +222,7 @@ smart server reply: S: 0000 The client may send Extra Parameters (see -Documentation/technical/pack-protocol.txt) as a colon-separated string +linkgit:gitprotocol-pack[5]) as a colon-separated string in the Git-Protocol HTTP header. Uses the `--http-backend-info-refs` option to @@ -518,5 +518,5 @@ References http://www.ietf.org/rfc/rfc1738.txt[RFC 1738: Uniform Resource Locators (URL)] http://www.ietf.org/rfc/rfc2616.txt[RFC 2616: Hypertext Transfer Protocol -- HTTP/1.1] -link:technical/pack-protocol.html -link:technical/protocol-capabilities.html +linkgit:gitprotocol-pack[5] +linkgit:gitprotocol-capabilities[5] diff --git a/Documentation/technical/long-running-process-protocol.txt b/Documentation/technical/long-running-process-protocol.txt index aa0aa9af1c..6f33654b42 100644 --- a/Documentation/technical/long-running-process-protocol.txt +++ b/Documentation/technical/long-running-process-protocol.txt @@ -3,7 +3,7 @@ Long-running process protocol This protocol is used when Git needs to communicate with an external process throughout the entire life of a single Git command. All -communication is in pkt-line format (see technical/protocol-common.txt) +communication is in pkt-line format (see linkgit:gitprotocol-common[5]) over standard input and standard output. Handshake diff --git a/Documentation/technical/pack-protocol.txt b/Documentation/technical/pack-protocol.txt deleted file mode 100644 index e13a2c064d..0000000000 --- a/Documentation/technical/pack-protocol.txt +++ /dev/null @@ -1,709 +0,0 @@ -Packfile transfer protocols -=========================== - -Git supports transferring data in packfiles over the ssh://, git://, http:// and -file:// transports. There exist two sets of protocols, one for pushing -data from a client to a server and another for fetching data from a -server to a client. The three transports (ssh, git, file) use the same -protocol to transfer data. http is documented in http-protocol.txt. - -The processes invoked in the canonical Git implementation are 'upload-pack' -on the server side and 'fetch-pack' on the client side for fetching data; -then 'receive-pack' on the server and 'send-pack' on the client for pushing -data. The protocol functions to have a server tell a client what is -currently on the server, then for the two to negotiate the smallest amount -of data to send in order to fully update one or the other. - -pkt-line Format ---------------- - -The descriptions below build on the pkt-line format described in -protocol-common.txt. When the grammar indicate `PKT-LINE(...)`, unless -otherwise noted the usual pkt-line LF rules apply: the sender SHOULD -include a LF, but the receiver MUST NOT complain if it is not present. - -An error packet is a special pkt-line that contains an error string. - ----- - error-line = PKT-LINE("ERR" SP explanation-text) ----- - -Throughout the protocol, where `PKT-LINE(...)` is expected, an error packet MAY -be sent. Once this packet is sent by a client or a server, the data transfer -process defined in this protocol is terminated. - -Transports ----------- -There are three transports over which the packfile protocol is -initiated. The Git transport is a simple, unauthenticated server that -takes the command (almost always 'upload-pack', though Git -servers can be configured to be globally writable, in which 'receive- -pack' initiation is also allowed) with which the client wishes to -communicate and executes it and connects it to the requesting -process. - -In the SSH transport, the client just runs the 'upload-pack' -or 'receive-pack' process on the server over the SSH protocol and then -communicates with that invoked process over the SSH connection. - -The file:// transport runs the 'upload-pack' or 'receive-pack' -process locally and communicates with it over a pipe. - -Extra Parameters ----------------- - -The protocol provides a mechanism in which clients can send additional -information in its first message to the server. These are called "Extra -Parameters", and are supported by the Git, SSH, and HTTP protocols. - -Each Extra Parameter takes the form of `<key>=<value>` or `<key>`. - -Servers that receive any such Extra Parameters MUST ignore all -unrecognized keys. Currently, the only Extra Parameter recognized is -"version" with a value of '1' or '2'. See protocol-v2.txt for more -information on protocol version 2. - -Git Transport -------------- - -The Git transport starts off by sending the command and repository -on the wire using the pkt-line format, followed by a NUL byte and a -hostname parameter, terminated by a NUL byte. - - 0033git-upload-pack /project.git\0host=myserver.com\0 - -The transport may send Extra Parameters by adding an additional NUL -byte, and then adding one or more NUL-terminated strings: - - 003egit-upload-pack /project.git\0host=myserver.com\0\0version=1\0 - --- - git-proto-request = request-command SP pathname NUL - [ host-parameter NUL ] [ NUL extra-parameters ] - request-command = "git-upload-pack" / "git-receive-pack" / - "git-upload-archive" ; case sensitive - pathname = *( %x01-ff ) ; exclude NUL - host-parameter = "host=" hostname [ ":" port ] - extra-parameters = 1*extra-parameter - extra-parameter = 1*( %x01-ff ) NUL --- - -host-parameter is used for the -git-daemon name based virtual hosting. See --interpolated-path -option to git daemon, with the %H/%CH format characters. - -Basically what the Git client is doing to connect to an 'upload-pack' -process on the server side over the Git protocol is this: - - $ echo -e -n \ - "003agit-upload-pack /schacon/gitbook.git\0host=example.com\0" | - nc -v example.com 9418 - - -SSH Transport -------------- - -Initiating the upload-pack or receive-pack processes over SSH is -executing the binary on the server via SSH remote execution. -It is basically equivalent to running this: - - $ ssh git.example.com "git-upload-pack '/project.git'" - -For a server to support Git pushing and pulling for a given user over -SSH, that user needs to be able to execute one or both of those -commands via the SSH shell that they are provided on login. On some -systems, that shell access is limited to only being able to run those -two commands, or even just one of them. - -In an ssh:// format URI, it's absolute in the URI, so the '/' after -the host name (or port number) is sent as an argument, which is then -read by the remote git-upload-pack exactly as is, so it's effectively -an absolute path in the remote filesystem. - - git clone ssh://user@example.com/project.git - | - v - ssh user@example.com "git-upload-pack '/project.git'" - -In a "user@host:path" format URI, its relative to the user's home -directory, because the Git client will run: - - git clone user@example.com:project.git - | - v - ssh user@example.com "git-upload-pack 'project.git'" - -The exception is if a '~' is used, in which case -we execute it without the leading '/'. - - ssh://user@example.com/~alice/project.git, - | - v - ssh user@example.com "git-upload-pack '~alice/project.git'" - -Depending on the value of the `protocol.version` configuration variable, -Git may attempt to send Extra Parameters as a colon-separated string in -the GIT_PROTOCOL environment variable. This is done only if -the `ssh.variant` configuration variable indicates that the ssh command -supports passing environment variables as an argument. - -A few things to remember here: - -- The "command name" is spelled with dash (e.g. git-upload-pack), but - this can be overridden by the client; - -- The repository path is always quoted with single quotes. - -Fetching Data From a Server ---------------------------- - -When one Git repository wants to get data that a second repository -has, the first can 'fetch' from the second. This operation determines -what data the server has that the client does not then streams that -data down to the client in packfile format. - - -Reference Discovery -------------------- - -When the client initially connects the server will immediately respond -with a version number (if "version=1" is sent as an Extra Parameter), -and a listing of each reference it has (all branches and tags) along -with the object name that each reference currently points to. - - $ echo -e -n "0045git-upload-pack /schacon/gitbook.git\0host=example.com\0\0version=1\0" | - nc -v example.com 9418 - 000eversion 1 - 00887217a7c7e582c46cec22a130adf4b9d7d950fba0 HEAD\0multi_ack thin-pack - side-band side-band-64k ofs-delta shallow no-progress include-tag - 00441d3fcd5ced445d1abc402225c0b8a1299641f497 refs/heads/integration - 003f7217a7c7e582c46cec22a130adf4b9d7d950fba0 refs/heads/master - 003cb88d2441cac0977faf98efc80305012112238d9d refs/tags/v0.9 - 003c525128480b96c89e6418b1e40909bf6c5b2d580f refs/tags/v1.0 - 003fe92df48743b7bc7d26bcaabfddde0a1e20cae47c refs/tags/v1.0^{} - 0000 - -The returned response is a pkt-line stream describing each ref and -its current value. The stream MUST be sorted by name according to -the C locale ordering. - -If HEAD is a valid ref, HEAD MUST appear as the first advertised -ref. If HEAD is not a valid ref, HEAD MUST NOT appear in the -advertisement list at all, but other refs may still appear. - -The stream MUST include capability declarations behind a NUL on the -first ref. The peeled value of a ref (that is "ref^{}") MUST be -immediately after the ref itself, if presented. A conforming server -MUST peel the ref if it's an annotated tag. - ----- - advertised-refs = *1("version 1") - (no-refs / list-of-refs) - *shallow - flush-pkt - - no-refs = PKT-LINE(zero-id SP "capabilities^{}" - NUL capability-list) - - list-of-refs = first-ref *other-ref - first-ref = PKT-LINE(obj-id SP refname - NUL capability-list) - - other-ref = PKT-LINE(other-tip / other-peeled) - other-tip = obj-id SP refname - other-peeled = obj-id SP refname "^{}" - - shallow = PKT-LINE("shallow" SP obj-id) - - capability-list = capability *(SP capability) - capability = 1*(LC_ALPHA / DIGIT / "-" / "_") - LC_ALPHA = %x61-7A ----- - -Server and client MUST use lowercase for obj-id, both MUST treat obj-id -as case-insensitive. - -See protocol-capabilities.txt for a list of allowed server capabilities -and descriptions. - -Packfile Negotiation --------------------- -After reference and capabilities discovery, the client can decide to -terminate the connection by sending a flush-pkt, telling the server it can -now gracefully terminate, and disconnect, when it does not need any pack -data. This can happen with the ls-remote command, and also can happen when -the client already is up to date. - -Otherwise, it enters the negotiation phase, where the client and -server determine what the minimal packfile necessary for transport is, -by telling the server what objects it wants, its shallow objects -(if any), and the maximum commit depth it wants (if any). The client -will also send a list of the capabilities it wants to be in effect, -out of what the server said it could do with the first 'want' line. - ----- - upload-request = want-list - *shallow-line - *1depth-request - [filter-request] - flush-pkt - - want-list = first-want - *additional-want - - shallow-line = PKT-LINE("shallow" SP obj-id) - - depth-request = PKT-LINE("deepen" SP depth) / - PKT-LINE("deepen-since" SP timestamp) / - PKT-LINE("deepen-not" SP ref) - - first-want = PKT-LINE("want" SP obj-id SP capability-list) - additional-want = PKT-LINE("want" SP obj-id) - - depth = 1*DIGIT - - filter-request = PKT-LINE("filter" SP filter-spec) ----- - -Clients MUST send all the obj-ids it wants from the reference -discovery phase as 'want' lines. Clients MUST send at least one -'want' command in the request body. Clients MUST NOT mention an -obj-id in a 'want' command which did not appear in the response -obtained through ref discovery. - -The client MUST write all obj-ids which it only has shallow copies -of (meaning that it does not have the parents of a commit) as -'shallow' lines so that the server is aware of the limitations of -the client's history. - -The client now sends the maximum commit history depth it wants for -this transaction, which is the number of commits it wants from the -tip of the history, if any, as a 'deepen' line. A depth of 0 is the -same as not making a depth request. The client does not want to receive -any commits beyond this depth, nor does it want objects needed only to -complete those commits. Commits whose parents are not received as a -result are defined as shallow and marked as such in the server. This -information is sent back to the client in the next step. - -The client can optionally request that pack-objects omit various -objects from the packfile using one of several filtering techniques. -These are intended for use with partial clone and partial fetch -operations. An object that does not meet a filter-spec value is -omitted unless explicitly requested in a 'want' line. See `rev-list` -for possible filter-spec values. - -Once all the 'want's and 'shallow's (and optional 'deepen') are -transferred, clients MUST send a flush-pkt, to tell the server side -that it is done sending the list. - -Otherwise, if the client sent a positive depth request, the server -will determine which commits will and will not be shallow and -send this information to the client. If the client did not request -a positive depth, this step is skipped. - ----- - shallow-update = *shallow-line - *unshallow-line - flush-pkt - - shallow-line = PKT-LINE("shallow" SP obj-id) - - unshallow-line = PKT-LINE("unshallow" SP obj-id) ----- - -If the client has requested a positive depth, the server will compute -the set of commits which are no deeper than the desired depth. The set -of commits start at the client's wants. - -The server writes 'shallow' lines for each -commit whose parents will not be sent as a result. The server writes -an 'unshallow' line for each commit which the client has indicated is -shallow, but is no longer shallow at the currently requested depth -(that is, its parents will now be sent). The server MUST NOT mark -as unshallow anything which the client has not indicated was shallow. - -Now the client will send a list of the obj-ids it has using 'have' -lines, so the server can make a packfile that only contains the objects -that the client needs. In multi_ack mode, the canonical implementation -will send up to 32 of these at a time, then will send a flush-pkt. The -canonical implementation will skip ahead and send the next 32 immediately, -so that there is always a block of 32 "in-flight on the wire" at a time. - ----- - upload-haves = have-list - compute-end - - have-list = *have-line - have-line = PKT-LINE("have" SP obj-id) - compute-end = flush-pkt / PKT-LINE("done") ----- - -If the server reads 'have' lines, it then will respond by ACKing any -of the obj-ids the client said it had that the server also has. The -server will ACK obj-ids differently depending on which ack mode is -chosen by the client. - -In multi_ack mode: - - * the server will respond with 'ACK obj-id continue' for any common - commits. - - * once the server has found an acceptable common base commit and is - ready to make a packfile, it will blindly ACK all 'have' obj-ids - back to the client. - - * the server will then send a 'NAK' and then wait for another response - from the client - either a 'done' or another list of 'have' lines. - -In multi_ack_detailed mode: - - * the server will differentiate the ACKs where it is signaling - that it is ready to send data with 'ACK obj-id ready' lines, and - signals the identified common commits with 'ACK obj-id common' lines. - -Without either multi_ack or multi_ack_detailed: - - * upload-pack sends "ACK obj-id" on the first common object it finds. - After that it says nothing until the client gives it a "done". - - * upload-pack sends "NAK" on a flush-pkt if no common object - has been found yet. If one has been found, and thus an ACK - was already sent, it's silent on the flush-pkt. - -After the client has gotten enough ACK responses that it can determine -that the server has enough information to send an efficient packfile -(in the canonical implementation, this is determined when it has received -enough ACKs that it can color everything left in the --date-order queue -as common with the server, or the --date-order queue is empty), or the -client determines that it wants to give up (in the canonical implementation, -this is determined when the client sends 256 'have' lines without getting -any of them ACKed by the server - meaning there is nothing in common and -the server should just send all of its objects), then the client will send -a 'done' command. The 'done' command signals to the server that the client -is ready to receive its packfile data. - -However, the 256 limit *only* turns on in the canonical client -implementation if we have received at least one "ACK %s continue" -during a prior round. This helps to ensure that at least one common -ancestor is found before we give up entirely. - -Once the 'done' line is read from the client, the server will either -send a final 'ACK obj-id' or it will send a 'NAK'. 'obj-id' is the object -name of the last commit determined to be common. The server only sends -ACK after 'done' if there is at least one common base and multi_ack or -multi_ack_detailed is enabled. The server always sends NAK after 'done' -if there is no common base found. - -Instead of 'ACK' or 'NAK', the server may send an error message (for -example, if it does not recognize an object in a 'want' line received -from the client). - -Then the server will start sending its packfile data. - ----- - server-response = *ack_multi ack / nak - ack_multi = PKT-LINE("ACK" SP obj-id ack_status) - ack_status = "continue" / "common" / "ready" - ack = PKT-LINE("ACK" SP obj-id) - nak = PKT-LINE("NAK") ----- - -A simple clone may look like this (with no 'have' lines): - ----- - C: 0054want 74730d410fcb6603ace96f1dc55ea6196122532d multi_ack \ - side-band-64k ofs-delta\n - C: 0032want 7d1665144a3a975c05f1f43902ddaf084e784dbe\n - C: 0032want 5a3f6be755bbb7deae50065988cbfa1ffa9ab68a\n - C: 0032want 7e47fe2bd8d01d481f44d7af0531bd93d3b21c01\n - C: 0032want 74730d410fcb6603ace96f1dc55ea6196122532d\n - C: 0000 - C: 0009done\n - - S: 0008NAK\n - S: [PACKFILE] ----- - -An incremental update (fetch) response might look like this: - ----- - C: 0054want 74730d410fcb6603ace96f1dc55ea6196122532d multi_ack \ - side-band-64k ofs-delta\n - C: 0032want 7d1665144a3a975c05f1f43902ddaf084e784dbe\n - C: 0032want 5a3f6be755bbb7deae50065988cbfa1ffa9ab68a\n - C: 0000 - C: 0032have 7e47fe2bd8d01d481f44d7af0531bd93d3b21c01\n - C: [30 more have lines] - C: 0032have 74730d410fcb6603ace96f1dc55ea6196122532d\n - C: 0000 - - S: 003aACK 7e47fe2bd8d01d481f44d7af0531bd93d3b21c01 continue\n - S: 003aACK 74730d410fcb6603ace96f1dc55ea6196122532d continue\n - S: 0008NAK\n - - C: 0009done\n - - S: 0031ACK 74730d410fcb6603ace96f1dc55ea6196122532d\n - S: [PACKFILE] ----- - - -Packfile Data -------------- - -Now that the client and server have finished negotiation about what -the minimal amount of data that needs to be sent to the client is, the server -will construct and send the required data in packfile format. - -See pack-format.txt for what the packfile itself actually looks like. - -If 'side-band' or 'side-band-64k' capabilities have been specified by -the client, the server will send the packfile data multiplexed. - -Each packet starting with the packet-line length of the amount of data -that follows, followed by a single byte specifying the sideband the -following data is coming in on. - -In 'side-band' mode, it will send up to 999 data bytes plus 1 control -code, for a total of up to 1000 bytes in a pkt-line. In 'side-band-64k' -mode it will send up to 65519 data bytes plus 1 control code, for a -total of up to 65520 bytes in a pkt-line. - -The sideband byte will be a '1', '2' or a '3'. Sideband '1' will contain -packfile data, sideband '2' will be used for progress information that the -client will generally print to stderr and sideband '3' is used for error -information. - -If no 'side-band' capability was specified, the server will stream the -entire packfile without multiplexing. - - -Pushing Data To a Server ------------------------- - -Pushing data to a server will invoke the 'receive-pack' process on the -server, which will allow the client to tell it which references it should -update and then send all the data the server will need for those new -references to be complete. Once all the data is received and validated, -the server will then update its references to what the client specified. - -Authentication --------------- - -The protocol itself contains no authentication mechanisms. That is to be -handled by the transport, such as SSH, before the 'receive-pack' process is -invoked. If 'receive-pack' is configured over the Git transport, those -repositories will be writable by anyone who can access that port (9418) as -that transport is unauthenticated. - -Reference Discovery -------------------- - -The reference discovery phase is done nearly the same way as it is in the -fetching protocol. Each reference obj-id and name on the server is sent -in packet-line format to the client, followed by a flush-pkt. The only -real difference is that the capability listing is different - the only -possible values are 'report-status', 'report-status-v2', 'delete-refs', -'ofs-delta', 'atomic' and 'push-options'. - -Reference Update Request and Packfile Transfer ----------------------------------------------- - -Once the client knows what references the server is at, it can send a -list of reference update requests. For each reference on the server -that it wants to update, it sends a line listing the obj-id currently on -the server, the obj-id the client would like to update it to and the name -of the reference. - -This list is followed by a flush-pkt. - ----- - update-requests = *shallow ( command-list | push-cert ) - - shallow = PKT-LINE("shallow" SP obj-id) - - command-list = PKT-LINE(command NUL capability-list) - *PKT-LINE(command) - flush-pkt - - command = create / delete / update - create = zero-id SP new-id SP name - delete = old-id SP zero-id SP name - update = old-id SP new-id SP name - - old-id = obj-id - new-id = obj-id - - push-cert = PKT-LINE("push-cert" NUL capability-list LF) - PKT-LINE("certificate version 0.1" LF) - PKT-LINE("pusher" SP ident LF) - PKT-LINE("pushee" SP url LF) - PKT-LINE("nonce" SP nonce LF) - *PKT-LINE("push-option" SP push-option LF) - PKT-LINE(LF) - *PKT-LINE(command LF) - *PKT-LINE(gpg-signature-lines LF) - PKT-LINE("push-cert-end" LF) - - push-option = 1*( VCHAR | SP ) ----- - -If the server has advertised the 'push-options' capability and the client has -specified 'push-options' as part of the capability list above, the client then -sends its push options followed by a flush-pkt. - ----- - push-options = *PKT-LINE(push-option) flush-pkt ----- - -For backwards compatibility with older Git servers, if the client sends a push -cert and push options, it MUST send its push options both embedded within the -push cert and after the push cert. (Note that the push options within the cert -are prefixed, but the push options after the cert are not.) Both these lists -MUST be the same, modulo the prefix. - -After that the packfile that -should contain all the objects that the server will need to complete the new -references will be sent. - ----- - packfile = "PACK" 28*(OCTET) ----- - -If the receiving end does not support delete-refs, the sending end MUST -NOT ask for delete command. - -If the receiving end does not support push-cert, the sending end -MUST NOT send a push-cert command. When a push-cert command is -sent, command-list MUST NOT be sent; the commands recorded in the -push certificate is used instead. - -The packfile MUST NOT be sent if the only command used is 'delete'. - -A packfile MUST be sent if either create or update command is used, -even if the server already has all the necessary objects. In this -case the client MUST send an empty packfile. The only time this -is likely to happen is if the client is creating -a new branch or a tag that points to an existing obj-id. - -The server will receive the packfile, unpack it, then validate each -reference that is being updated that it hasn't changed while the request -was being processed (the obj-id is still the same as the old-id), and -it will run any update hooks to make sure that the update is acceptable. -If all of that is fine, the server will then update the references. - -Push Certificate ----------------- - -A push certificate begins with a set of header lines. After the -header and an empty line, the protocol commands follow, one per -line. Note that the trailing LF in push-cert PKT-LINEs is _not_ -optional; it must be present. - -Currently, the following header fields are defined: - -`pusher` ident:: - Identify the GPG key in "Human Readable Name <email@address>" - format. - -`pushee` url:: - The repository URL (anonymized, if the URL contains - authentication material) the user who ran `git push` - intended to push into. - -`nonce` nonce:: - The 'nonce' string the receiving repository asked the - pushing user to include in the certificate, to prevent - replay attacks. - -The GPG signature lines are a detached signature for the contents -recorded in the push certificate before the signature block begins. -The detached signature is used to certify that the commands were -given by the pusher, who must be the signer. - -Report Status -------------- - -After receiving the pack data from the sender, the receiver sends a -report if 'report-status' or 'report-status-v2' capability is in effect. -It is a short listing of what happened in that update. It will first -list the status of the packfile unpacking as either 'unpack ok' or -'unpack [error]'. Then it will list the status for each of the references -that it tried to update. Each line is either 'ok [refname]' if the -update was successful, or 'ng [refname] [error]' if the update was not. - ----- - report-status = unpack-status - 1*(command-status) - flush-pkt - - unpack-status = PKT-LINE("unpack" SP unpack-result) - unpack-result = "ok" / error-msg - - command-status = command-ok / command-fail - command-ok = PKT-LINE("ok" SP refname) - command-fail = PKT-LINE("ng" SP refname SP error-msg) - - error-msg = 1*(OCTET) ; where not "ok" ----- - -The 'report-status-v2' capability extends the protocol by adding new option -lines in order to support reporting of reference rewritten by the -'proc-receive' hook. The 'proc-receive' hook may handle a command for a -pseudo-reference which may create or update one or more references, and each -reference may have different name, different new-oid, and different old-oid. - ----- - report-status-v2 = unpack-status - 1*(command-status-v2) - flush-pkt - - unpack-status = PKT-LINE("unpack" SP unpack-result) - unpack-result = "ok" / error-msg - - command-status-v2 = command-ok-v2 / command-fail - command-ok-v2 = command-ok - *option-line - - command-ok = PKT-LINE("ok" SP refname) - command-fail = PKT-LINE("ng" SP refname SP error-msg) - - error-msg = 1*(OCTET) ; where not "ok" - - option-line = *1(option-refname) - *1(option-old-oid) - *1(option-new-oid) - *1(option-forced-update) - - option-refname = PKT-LINE("option" SP "refname" SP refname) - option-old-oid = PKT-LINE("option" SP "old-oid" SP obj-id) - option-new-oid = PKT-LINE("option" SP "new-oid" SP obj-id) - option-force = PKT-LINE("option" SP "forced-update") - ----- - -Updates can be unsuccessful for a number of reasons. The reference can have -changed since the reference discovery phase was originally sent, meaning -someone pushed in the meantime. The reference being pushed could be a -non-fast-forward reference and the update hooks or configuration could be -set to not allow that, etc. Also, some references can be updated while others -can be rejected. - -An example client/server communication might look like this: - ----- - S: 006274730d410fcb6603ace96f1dc55ea6196122532d refs/heads/local\0report-status delete-refs ofs-delta\n - S: 003e7d1665144a3a975c05f1f43902ddaf084e784dbe refs/heads/debug\n - S: 003f74730d410fcb6603ace96f1dc55ea6196122532d refs/heads/master\n - S: 003d74730d410fcb6603ace96f1dc55ea6196122532d refs/heads/team\n - S: 0000 - - C: 00677d1665144a3a975c05f1f43902ddaf084e784dbe 74730d410fcb6603ace96f1dc55ea6196122532d refs/heads/debug\n - C: 006874730d410fcb6603ace96f1dc55ea6196122532d 5a3f6be755bbb7deae50065988cbfa1ffa9ab68a refs/heads/master\n - C: 0000 - C: [PACKDATA] - - S: 000eunpack ok\n - S: 0018ok refs/heads/debug\n - S: 002ang refs/heads/master non-fast-forward\n ----- diff --git a/Documentation/technical/packfile-uri.txt b/Documentation/technical/packfile-uri.txt index 1eb525fe76..9d453d4765 100644 --- a/Documentation/technical/packfile-uri.txt +++ b/Documentation/technical/packfile-uri.txt @@ -18,7 +18,7 @@ a `packfile-uris` argument, the server MAY send a `packfile-uris` section directly before the `packfile` section (right after `wanted-refs` if it is sent) containing URIs of any of the given protocols. The URIs point to packfiles that use only features that the client has declared that it supports -(e.g. ofs-delta and thin-pack). See protocol-v2.txt for the documentation of +(e.g. ofs-delta and thin-pack). See linkgit:gitprotocol-v2[5] for the documentation of this section. Clients should then download and index all the given URIs (in addition to diff --git a/Documentation/technical/partial-clone.txt b/Documentation/technical/partial-clone.txt index 99f0eb3040..92fcee2bff 100644 --- a/Documentation/technical/partial-clone.txt +++ b/Documentation/technical/partial-clone.txt @@ -79,7 +79,7 @@ Design Details upload-pack negotiation. + This uses the existing capability discovery mechanism. -See "filter" in Documentation/technical/pack-protocol.txt. +See "filter" in linkgit:gitprotocol-pack[5]. - Clients pass a "filter-spec" to clone and fetch which is passed to the server to request filtering during packfile construction. diff --git a/Documentation/technical/protocol-capabilities.txt b/Documentation/technical/protocol-capabilities.txt deleted file mode 100644 index 9dfade930d..0000000000 --- a/Documentation/technical/protocol-capabilities.txt +++ /dev/null @@ -1,380 +0,0 @@ -Git Protocol Capabilities -========================= - -NOTE: this document describes capabilities for versions 0 and 1 of the pack -protocol. For version 2, please refer to the link:protocol-v2.html[protocol-v2] -doc. - -Servers SHOULD support all capabilities defined in this document. - -On the very first line of the initial server response of either -receive-pack and upload-pack the first reference is followed by -a NUL byte and then a list of space delimited server capabilities. -These allow the server to declare what it can and cannot support -to the client. - -Client will then send a space separated list of capabilities it wants -to be in effect. The client MUST NOT ask for capabilities the server -did not say it supports. - -Server MUST diagnose and abort if capabilities it does not understand -was sent. Server MUST NOT ignore capabilities that client requested -and server advertised. As a consequence of these rules, server MUST -NOT advertise capabilities it does not understand. - -The 'atomic', 'report-status', 'report-status-v2', 'delete-refs', 'quiet', -and 'push-cert' capabilities are sent and recognized by the receive-pack -(push to server) process. - -The 'ofs-delta' and 'side-band-64k' capabilities are sent and recognized -by both upload-pack and receive-pack protocols. The 'agent' and 'session-id' -capabilities may optionally be sent in both protocols. - -All other capabilities are only recognized by the upload-pack (fetch -from server) process. - -multi_ack ---------- - -The 'multi_ack' capability allows the server to return "ACK obj-id -continue" as soon as it finds a commit that it can use as a common -base, between the client's wants and the client's have set. - -By sending this early, the server can potentially head off the client -from walking any further down that particular branch of the client's -repository history. The client may still need to walk down other -branches, sending have lines for those, until the server has a -complete cut across the DAG, or the client has said "done". - -Without multi_ack, a client sends have lines in --date-order until -the server has found a common base. That means the client will send -have lines that are already known by the server to be common, because -they overlap in time with another branch that the server hasn't found -a common base on yet. - -For example suppose the client has commits in caps that the server -doesn't and the server has commits in lower case that the client -doesn't, as in the following diagram: - - +---- u ---------------------- x - / +----- y - / / - a -- b -- c -- d -- E -- F - \ - +--- Q -- R -- S - -If the client wants x,y and starts out by saying have F,S, the server -doesn't know what F,S is. Eventually the client says "have d" and -the server sends "ACK d continue" to let the client know to stop -walking down that line (so don't send c-b-a), but it's not done yet, -it needs a base for x. The client keeps going with S-R-Q, until a -gets reached, at which point the server has a clear base and it all -ends. - -Without multi_ack the client would have sent that c-b-a chain anyway, -interleaved with S-R-Q. - -multi_ack_detailed ------------------- -This is an extension of multi_ack that permits client to better -understand the server's in-memory state. See pack-protocol.txt, -section "Packfile Negotiation" for more information. - -no-done -------- -This capability should only be used with the smart HTTP protocol. If -multi_ack_detailed and no-done are both present, then the sender is -free to immediately send a pack following its first "ACK obj-id ready" -message. - -Without no-done in the smart HTTP protocol, the server session would -end and the client has to make another trip to send "done" before -the server can send the pack. no-done removes the last round and -thus slightly reduces latency. - -thin-pack ---------- - -A thin pack is one with deltas which reference base objects not -contained within the pack (but are known to exist at the receiving -end). This can reduce the network traffic significantly, but it -requires the receiving end to know how to "thicken" these packs by -adding the missing bases to the pack. - -The upload-pack server advertises 'thin-pack' when it can generate -and send a thin pack. A client requests the 'thin-pack' capability -when it understands how to "thicken" it, notifying the server that -it can receive such a pack. A client MUST NOT request the -'thin-pack' capability if it cannot turn a thin pack into a -self-contained pack. - -Receive-pack, on the other hand, is assumed by default to be able to -handle thin packs, but can ask the client not to use the feature by -advertising the 'no-thin' capability. A client MUST NOT send a thin -pack if the server advertises the 'no-thin' capability. - -The reasons for this asymmetry are historical. The receive-pack -program did not exist until after the invention of thin packs, so -historically the reference implementation of receive-pack always -understood thin packs. Adding 'no-thin' later allowed receive-pack -to disable the feature in a backwards-compatible manner. - - -side-band, side-band-64k ------------------------- - -This capability means that server can send, and client understand multiplexed -progress reports and error info interleaved with the packfile itself. - -These two options are mutually exclusive. A modern client always -favors 'side-band-64k'. - -Either mode indicates that the packfile data will be streamed broken -up into packets of up to either 1000 bytes in the case of 'side_band', -or 65520 bytes in the case of 'side_band_64k'. Each packet is made up -of a leading 4-byte pkt-line length of how much data is in the packet, -followed by a 1-byte stream code, followed by the actual data. - -The stream code can be one of: - - 1 - pack data - 2 - progress messages - 3 - fatal error message just before stream aborts - -The "side-band-64k" capability came about as a way for newer clients -that can handle much larger packets to request packets that are -actually crammed nearly full, while maintaining backward compatibility -for the older clients. - -Further, with side-band and its up to 1000-byte messages, it's actually -999 bytes of payload and 1 byte for the stream code. With side-band-64k, -same deal, you have up to 65519 bytes of data and 1 byte for the stream -code. - -The client MUST send only maximum of one of "side-band" and "side- -band-64k". Server MUST diagnose it as an error if client requests -both. - -ofs-delta ---------- - -Server can send, and client understand PACKv2 with delta referring to -its base by position in pack rather than by an obj-id. That is, they can -send/read OBJ_OFS_DELTA (aka type 6) in a packfile. - -agent ------ - -The server may optionally send a capability of the form `agent=X` to -notify the client that the server is running version `X`. The client may -optionally return its own agent string by responding with an `agent=Y` -capability (but it MUST NOT do so if the server did not mention the -agent capability). The `X` and `Y` strings may contain any printable -ASCII characters except space (i.e., the byte range 32 < x < 127), and -are typically of the form "package/version" (e.g., "git/1.8.3.1"). The -agent strings are purely informative for statistics and debugging -purposes, and MUST NOT be used to programmatically assume the presence -or absence of particular features. - -object-format -------------- - -This capability, which takes a hash algorithm as an argument, indicates -that the server supports the given hash algorithms. It may be sent -multiple times; if so, the first one given is the one used in the ref -advertisement. - -When provided by the client, this indicates that it intends to use the -given hash algorithm to communicate. The algorithm provided must be one -that the server supports. - -If this capability is not provided, it is assumed that the only -supported algorithm is SHA-1. - -symref ------- - -This parameterized capability is used to inform the receiver which symbolic ref -points to which ref; for example, "symref=HEAD:refs/heads/master" tells the -receiver that HEAD points to master. This capability can be repeated to -represent multiple symrefs. - -Servers SHOULD include this capability for the HEAD symref if it is one of the -refs being sent. - -Clients MAY use the parameters from this capability to select the proper initial -branch when cloning a repository. - -shallow -------- - -This capability adds "deepen", "shallow" and "unshallow" commands to -the fetch-pack/upload-pack protocol so clients can request shallow -clones. - -deepen-since ------------- - -This capability adds "deepen-since" command to fetch-pack/upload-pack -protocol so the client can request shallow clones that are cut at a -specific time, instead of depth. Internally it's equivalent of doing -"rev-list --max-age=<timestamp>" on the server side. "deepen-since" -cannot be used with "deepen". - -deepen-not ----------- - -This capability adds "deepen-not" command to fetch-pack/upload-pack -protocol so the client can request shallow clones that are cut at a -specific revision, instead of depth. Internally it's equivalent of -doing "rev-list --not <rev>" on the server side. "deepen-not" -cannot be used with "deepen", but can be used with "deepen-since". - -deepen-relative ---------------- - -If this capability is requested by the client, the semantics of -"deepen" command is changed. The "depth" argument is the depth from -the current shallow boundary, instead of the depth from remote refs. - -no-progress ------------ - -The client was started with "git clone -q" or something, and doesn't -want that side band 2. Basically the client just says "I do not -wish to receive stream 2 on sideband, so do not send it to me, and if -you did, I will drop it on the floor anyway". However, the sideband -channel 3 is still used for error responses. - -include-tag ------------ - -The 'include-tag' capability is about sending annotated tags if we are -sending objects they point to. If we pack an object to the client, and -a tag object points exactly at that object, we pack the tag object too. -In general this allows a client to get all new annotated tags when it -fetches a branch, in a single network connection. - -Clients MAY always send include-tag, hardcoding it into a request when -the server advertises this capability. The decision for a client to -request include-tag only has to do with the client's desires for tag -data, whether or not a server had advertised objects in the -refs/tags/* namespace. - -Servers MUST pack the tags if their referrant is packed and the client -has requested include-tags. - -Clients MUST be prepared for the case where a server has ignored -include-tag and has not actually sent tags in the pack. In such -cases the client SHOULD issue a subsequent fetch to acquire the tags -that include-tag would have otherwise given the client. - -The server SHOULD send include-tag, if it supports it, regardless -of whether or not there are tags available. - -report-status -------------- - -The receive-pack process can receive a 'report-status' capability, -which tells it that the client wants a report of what happened after -a packfile upload and reference update. If the pushing client requests -this capability, after unpacking and updating references the server -will respond with whether the packfile unpacked successfully and if -each reference was updated successfully. If any of those were not -successful, it will send back an error message. See pack-protocol.txt -for example messages. - -report-status-v2 ----------------- - -Capability 'report-status-v2' extends capability 'report-status' by -adding new "option" directives in order to support reference rewritten by -the "proc-receive" hook. The "proc-receive" hook may handle a command -for a pseudo-reference which may create or update a reference with -different name, new-oid, and old-oid. While the capability -'report-status' cannot report for such case. See pack-protocol.txt -for details. - -delete-refs ------------ - -If the server sends back the 'delete-refs' capability, it means that -it is capable of accepting a zero-id value as the target -value of a reference update. It is not sent back by the client, it -simply informs the client that it can be sent zero-id values -to delete references. - -quiet ------ - -If the receive-pack server advertises the 'quiet' capability, it is -capable of silencing human-readable progress output which otherwise may -be shown when processing the received pack. A send-pack client should -respond with the 'quiet' capability to suppress server-side progress -reporting if the local progress reporting is also being suppressed -(e.g., via `push -q`, or if stderr does not go to a tty). - -atomic ------- - -If the server sends the 'atomic' capability it is capable of accepting -atomic pushes. If the pushing client requests this capability, the server -will update the refs in one atomic transaction. Either all refs are -updated or none. - -push-options ------------- - -If the server sends the 'push-options' capability it is able to accept -push options after the update commands have been sent, but before the -packfile is streamed. If the pushing client requests this capability, -the server will pass the options to the pre- and post- receive hooks -that process this push request. - -allow-tip-sha1-in-want ----------------------- - -If the upload-pack server advertises this capability, fetch-pack may -send "want" lines with object names that exist at the server but are not -advertised by upload-pack. For historical reasons, the name of this -capability contains "sha1". Object names are always given using the -object format negotiated through the 'object-format' capability. - -allow-reachable-sha1-in-want ----------------------------- - -If the upload-pack server advertises this capability, fetch-pack may -send "want" lines with object names that exist at the server but are not -advertised by upload-pack. For historical reasons, the name of this -capability contains "sha1". Object names are always given using the -object format negotiated through the 'object-format' capability. - -push-cert=<nonce> ------------------ - -The receive-pack server that advertises this capability is willing -to accept a signed push certificate, and asks the <nonce> to be -included in the push certificate. A send-pack client MUST NOT -send a push-cert packet unless the receive-pack server advertises -this capability. - -filter ------- - -If the upload-pack server advertises the 'filter' capability, -fetch-pack may send "filter" commands to request a partial clone -or partial fetch and request that the server omit various objects -from the packfile. - -session-id=<session id> ------------------------ - -The server may advertise a session ID that can be used to identify this process -across multiple requests. The client may advertise its own session ID back to -the server as well. - -Session IDs should be unique to a given process. They must fit within a -packet-line, and must not contain non-printable or whitespace characters. The -current implementation uses trace2 session IDs (see -link:api-trace2.html[api-trace2] for details), but this may change and users of -the session ID should not rely on this fact. diff --git a/Documentation/technical/protocol-common.txt b/Documentation/technical/protocol-common.txt deleted file mode 100644 index ecedb34bba..0000000000 --- a/Documentation/technical/protocol-common.txt +++ /dev/null @@ -1,99 +0,0 @@ -Documentation Common to Pack and Http Protocols -=============================================== - -ABNF Notation -------------- - -ABNF notation as described by RFC 5234 is used within the protocol documents, -except the following replacement core rules are used: ----- - HEXDIG = DIGIT / "a" / "b" / "c" / "d" / "e" / "f" ----- - -We also define the following common rules: ----- - NUL = %x00 - zero-id = 40*"0" - obj-id = 40*(HEXDIGIT) - - refname = "HEAD" - refname /= "refs/" <see discussion below> ----- - -A refname is a hierarchical octet string beginning with "refs/" and -not violating the 'git-check-ref-format' command's validation rules. -More specifically, they: - -. They can include slash `/` for hierarchical (directory) - grouping, but no slash-separated component can begin with a - dot `.`. - -. They must contain at least one `/`. This enforces the presence of a - category like `heads/`, `tags/` etc. but the actual names are not - restricted. - -. They cannot have two consecutive dots `..` anywhere. - -. They cannot have ASCII control characters (i.e. bytes whose - values are lower than \040, or \177 `DEL`), space, tilde `~`, - caret `^`, colon `:`, question-mark `?`, asterisk `*`, - or open bracket `[` anywhere. - -. They cannot end with a slash `/` or a dot `.`. - -. They cannot end with the sequence `.lock`. - -. They cannot contain a sequence `@{`. - -. They cannot contain a `\\`. - - -pkt-line Format ---------------- - -Much (but not all) of the payload is described around pkt-lines. - -A pkt-line is a variable length binary string. The first four bytes -of the line, the pkt-len, indicates the total length of the line, -in hexadecimal. The pkt-len includes the 4 bytes used to contain -the length's hexadecimal representation. - -A pkt-line MAY contain binary data, so implementors MUST ensure -pkt-line parsing/formatting routines are 8-bit clean. - -A non-binary line SHOULD BE terminated by an LF, which if present -MUST be included in the total length. Receivers MUST treat pkt-lines -with non-binary data the same whether or not they contain the trailing -LF (stripping the LF if present, and not complaining when it is -missing). - -The maximum length of a pkt-line's data component is 65516 bytes. -Implementations MUST NOT send pkt-line whose length exceeds 65520 -(65516 bytes of payload + 4 bytes of length data). - -Implementations SHOULD NOT send an empty pkt-line ("0004"). - -A pkt-line with a length field of 0 ("0000"), called a flush-pkt, -is a special case and MUST be handled differently than an empty -pkt-line ("0004"). - ----- - pkt-line = data-pkt / flush-pkt - - data-pkt = pkt-len pkt-payload - pkt-len = 4*(HEXDIG) - pkt-payload = (pkt-len - 4)*(OCTET) - - flush-pkt = "0000" ----- - -Examples (as C-style strings): - ----- - pkt-line actual value - --------------------------------- - "0006a\n" "a\n" - "0005a" "a" - "000bfoobar\n" "foobar\n" - "0004" "" ----- diff --git a/Documentation/technical/protocol-v2.txt b/Documentation/technical/protocol-v2.txt deleted file mode 100644 index 8a877d27e2..0000000000 --- a/Documentation/technical/protocol-v2.txt +++ /dev/null @@ -1,568 +0,0 @@ -Git Wire Protocol, Version 2 -============================ - -This document presents a specification for a version 2 of Git's wire -protocol. Protocol v2 will improve upon v1 in the following ways: - - * Instead of multiple service names, multiple commands will be - supported by a single service - * Easily extendable as capabilities are moved into their own section - of the protocol, no longer being hidden behind a NUL byte and - limited by the size of a pkt-line - * Separate out other information hidden behind NUL bytes (e.g. agent - string as a capability and symrefs can be requested using 'ls-refs') - * Reference advertisement will be omitted unless explicitly requested - * ls-refs command to explicitly request some refs - * Designed with http and stateless-rpc in mind. With clear flush - semantics the http remote helper can simply act as a proxy - -In protocol v2 communication is command oriented. When first contacting a -server a list of capabilities will advertised. Some of these capabilities -will be commands which a client can request be executed. Once a command -has completed, a client can reuse the connection and request that other -commands be executed. - -Packet-Line Framing -------------------- - -All communication is done using packet-line framing, just as in v1. See -`Documentation/technical/pack-protocol.txt` and -`Documentation/technical/protocol-common.txt` for more information. - -In protocol v2 these special packets will have the following semantics: - - * '0000' Flush Packet (flush-pkt) - indicates the end of a message - * '0001' Delimiter Packet (delim-pkt) - separates sections of a message - * '0002' Response End Packet (response-end-pkt) - indicates the end of a - response for stateless connections - -Initial Client Request ----------------------- - -In general a client can request to speak protocol v2 by sending -`version=2` through the respective side-channel for the transport being -used which inevitably sets `GIT_PROTOCOL`. More information can be -found in `pack-protocol.txt` and `http-protocol.txt`, as well as the -`GIT_PROTOCOL` definition in `git.txt`. In all cases the -response from the server is the capability advertisement. - -Git Transport -~~~~~~~~~~~~~ - -When using the git:// transport, you can request to use protocol v2 by -sending "version=2" as an extra parameter: - - 003egit-upload-pack /project.git\0host=myserver.com\0\0version=2\0 - -SSH and File Transport -~~~~~~~~~~~~~~~~~~~~~~ - -When using either the ssh:// or file:// transport, the GIT_PROTOCOL -environment variable must be set explicitly to include "version=2". -The server may need to be configured to allow this environment variable -to pass. - -HTTP Transport -~~~~~~~~~~~~~~ - -When using the http:// or https:// transport a client makes a "smart" -info/refs request as described in `http-protocol.txt` and requests that -v2 be used by supplying "version=2" in the `Git-Protocol` header. - - C: GET $GIT_URL/info/refs?service=git-upload-pack HTTP/1.0 - C: Git-Protocol: version=2 - -A v2 server would reply: - - S: 200 OK - S: <Some headers> - S: ... - S: - S: 000eversion 2\n - S: <capability-advertisement> - -Subsequent requests are then made directly to the service -`$GIT_URL/git-upload-pack`. (This works the same for git-receive-pack). - -Uses the `--http-backend-info-refs` option to -linkgit:git-upload-pack[1]. - -The server may need to be configured to pass this header's contents via -the `GIT_PROTOCOL` variable. See the discussion in `git-http-backend.txt`. - -Capability Advertisement ------------------------- - -A server which decides to communicate (based on a request from a client) -using protocol version 2, notifies the client by sending a version string -in its initial response followed by an advertisement of its capabilities. -Each capability is a key with an optional value. Clients must ignore all -unknown keys. Semantics of unknown values are left to the definition of -each key. Some capabilities will describe commands which can be requested -to be executed by the client. - - capability-advertisement = protocol-version - capability-list - flush-pkt - - protocol-version = PKT-LINE("version 2" LF) - capability-list = *capability - capability = PKT-LINE(key[=value] LF) - - key = 1*(ALPHA | DIGIT | "-_") - value = 1*(ALPHA | DIGIT | " -_.,?\/{}[]()<>!@#$%^&*+=:;") - -Command Request ---------------- - -After receiving the capability advertisement, a client can then issue a -request to select the command it wants with any particular capabilities -or arguments. There is then an optional section where the client can -provide any command specific parameters or queries. Only a single -command can be requested at a time. - - request = empty-request | command-request - empty-request = flush-pkt - command-request = command - capability-list - delim-pkt - command-args - flush-pkt - command = PKT-LINE("command=" key LF) - command-args = *command-specific-arg - - command-specific-args are packet line framed arguments defined by - each individual command. - -The server will then check to ensure that the client's request is -comprised of a valid command as well as valid capabilities which were -advertised. If the request is valid the server will then execute the -command. A server MUST wait till it has received the client's entire -request before issuing a response. The format of the response is -determined by the command being executed, but in all cases a flush-pkt -indicates the end of the response. - -When a command has finished, and the client has received the entire -response from the server, a client can either request that another -command be executed or can terminate the connection. A client may -optionally send an empty request consisting of just a flush-pkt to -indicate that no more requests will be made. - -Capabilities ------------- - -There are two different types of capabilities: normal capabilities, -which can be used to convey information or alter the behavior of a -request, and commands, which are the core actions that a client wants to -perform (fetch, push, etc). - -Protocol version 2 is stateless by default. This means that all commands -must only last a single round and be stateless from the perspective of the -server side, unless the client has requested a capability indicating that -state should be maintained by the server. Clients MUST NOT require state -management on the server side in order to function correctly. This -permits simple round-robin load-balancing on the server side, without -needing to worry about state management. - -agent -~~~~~ - -The server can advertise the `agent` capability with a value `X` (in the -form `agent=X`) to notify the client that the server is running version -`X`. The client may optionally send its own agent string by including -the `agent` capability with a value `Y` (in the form `agent=Y`) in its -request to the server (but it MUST NOT do so if the server did not -advertise the agent capability). The `X` and `Y` strings may contain any -printable ASCII characters except space (i.e., the byte range 32 < x < -127), and are typically of the form "package/version" (e.g., -"git/1.8.3.1"). The agent strings are purely informative for statistics -and debugging purposes, and MUST NOT be used to programmatically assume -the presence or absence of particular features. - -ls-refs -~~~~~~~ - -`ls-refs` is the command used to request a reference advertisement in v2. -Unlike the current reference advertisement, ls-refs takes in arguments -which can be used to limit the refs sent from the server. - -Additional features not supported in the base command will be advertised -as the value of the command in the capability advertisement in the form -of a space separated list of features: "<command>=<feature 1> <feature 2>" - -ls-refs takes in the following arguments: - - symrefs - In addition to the object pointed by it, show the underlying ref - pointed by it when showing a symbolic ref. - peel - Show peeled tags. - ref-prefix <prefix> - When specified, only references having a prefix matching one of - the provided prefixes are displayed. Multiple instances may be - given, in which case references matching any prefix will be - shown. Note that this is purely for optimization; a server MAY - show refs not matching the prefix if it chooses, and clients - should filter the result themselves. - -If the 'unborn' feature is advertised the following argument can be -included in the client's request. - - unborn - The server will send information about HEAD even if it is a symref - pointing to an unborn branch in the form "unborn HEAD - symref-target:<target>". - -The output of ls-refs is as follows: - - output = *ref - flush-pkt - obj-id-or-unborn = (obj-id | "unborn") - ref = PKT-LINE(obj-id-or-unborn SP refname *(SP ref-attribute) LF) - ref-attribute = (symref | peeled) - symref = "symref-target:" symref-target - peeled = "peeled:" obj-id - -fetch -~~~~~ - -`fetch` is the command used to fetch a packfile in v2. It can be looked -at as a modified version of the v1 fetch where the ref-advertisement is -stripped out (since the `ls-refs` command fills that role) and the -message format is tweaked to eliminate redundancies and permit easy -addition of future extensions. - -Additional features not supported in the base command will be advertised -as the value of the command in the capability advertisement in the form -of a space separated list of features: "<command>=<feature 1> <feature 2>" - -A `fetch` request can take the following arguments: - - want <oid> - Indicates to the server an object which the client wants to - retrieve. Wants can be anything and are not limited to - advertised objects. - - have <oid> - Indicates to the server an object which the client has locally. - This allows the server to make a packfile which only contains - the objects that the client needs. Multiple 'have' lines can be - supplied. - - done - Indicates to the server that negotiation should terminate (or - not even begin if performing a clone) and that the server should - use the information supplied in the request to construct the - packfile. - - thin-pack - Request that a thin pack be sent, which is a pack with deltas - which reference base objects not contained within the pack (but - are known to exist at the receiving end). This can reduce the - network traffic significantly, but it requires the receiving end - to know how to "thicken" these packs by adding the missing bases - to the pack. - - no-progress - Request that progress information that would normally be sent on - side-band channel 2, during the packfile transfer, should not be - sent. However, the side-band channel 3 is still used for error - responses. - - include-tag - Request that annotated tags should be sent if the objects they - point to are being sent. - - ofs-delta - Indicate that the client understands PACKv2 with delta referring - to its base by position in pack rather than by an oid. That is, - they can read OBJ_OFS_DELTA (aka type 6) in a packfile. - -If the 'shallow' feature is advertised the following arguments can be -included in the clients request as well as the potential addition of the -'shallow-info' section in the server's response as explained below. - - shallow <oid> - A client must notify the server of all commits for which it only - has shallow copies (meaning that it doesn't have the parents of - a commit) by supplying a 'shallow <oid>' line for each such - object so that the server is aware of the limitations of the - client's history. This is so that the server is aware that the - client may not have all objects reachable from such commits. - - deepen <depth> - Requests that the fetch/clone should be shallow having a commit - depth of <depth> relative to the remote side. - - deepen-relative - Requests that the semantics of the "deepen" command be changed - to indicate that the depth requested is relative to the client's - current shallow boundary, instead of relative to the requested - commits. - - deepen-since <timestamp> - Requests that the shallow clone/fetch should be cut at a - specific time, instead of depth. Internally it's equivalent to - doing "git rev-list --max-age=<timestamp>". Cannot be used with - "deepen". - - deepen-not <rev> - Requests that the shallow clone/fetch should be cut at a - specific revision specified by '<rev>', instead of a depth. - Internally it's equivalent of doing "git rev-list --not <rev>". - Cannot be used with "deepen", but can be used with - "deepen-since". - -If the 'filter' feature is advertised, the following argument can be -included in the client's request: - - filter <filter-spec> - Request that various objects from the packfile be omitted - using one of several filtering techniques. These are intended - for use with partial clone and partial fetch operations. See - `rev-list` for possible "filter-spec" values. When communicating - with other processes, senders SHOULD translate scaled integers - (e.g. "1k") into a fully-expanded form (e.g. "1024") to aid - interoperability with older receivers that may not understand - newly-invented scaling suffixes. However, receivers SHOULD - accept the following suffixes: 'k', 'm', and 'g' for 1024, - 1048576, and 1073741824, respectively. - -If the 'ref-in-want' feature is advertised, the following argument can -be included in the client's request as well as the potential addition of -the 'wanted-refs' section in the server's response as explained below. - - want-ref <ref> - Indicates to the server that the client wants to retrieve a - particular ref, where <ref> is the full name of a ref on the - server. - -If the 'sideband-all' feature is advertised, the following argument can be -included in the client's request: - - sideband-all - Instruct the server to send the whole response multiplexed, not just - the packfile section. All non-flush and non-delim PKT-LINE in the - response (not only in the packfile section) will then start with a byte - indicating its sideband (1, 2, or 3), and the server may send "0005\2" - (a PKT-LINE of sideband 2 with no payload) as a keepalive packet. - -If the 'packfile-uris' feature is advertised, the following argument -can be included in the client's request as well as the potential -addition of the 'packfile-uris' section in the server's response as -explained below. - - packfile-uris <comma-separated list of protocols> - Indicates to the server that the client is willing to receive - URIs of any of the given protocols in place of objects in the - sent packfile. Before performing the connectivity check, the - client should download from all given URIs. Currently, the - protocols supported are "http" and "https". - -If the 'wait-for-done' feature is advertised, the following argument -can be included in the client's request. - - wait-for-done - Indicates to the server that it should never send "ready", but - should wait for the client to say "done" before sending the - packfile. - -The response of `fetch` is broken into a number of sections separated by -delimiter packets (0001), with each section beginning with its section -header. Most sections are sent only when the packfile is sent. - - output = acknowledgements flush-pkt | - [acknowledgments delim-pkt] [shallow-info delim-pkt] - [wanted-refs delim-pkt] [packfile-uris delim-pkt] - packfile flush-pkt - - acknowledgments = PKT-LINE("acknowledgments" LF) - (nak | *ack) - (ready) - ready = PKT-LINE("ready" LF) - nak = PKT-LINE("NAK" LF) - ack = PKT-LINE("ACK" SP obj-id LF) - - shallow-info = PKT-LINE("shallow-info" LF) - *PKT-LINE((shallow | unshallow) LF) - shallow = "shallow" SP obj-id - unshallow = "unshallow" SP obj-id - - wanted-refs = PKT-LINE("wanted-refs" LF) - *PKT-LINE(wanted-ref LF) - wanted-ref = obj-id SP refname - - packfile-uris = PKT-LINE("packfile-uris" LF) *packfile-uri - packfile-uri = PKT-LINE(40*(HEXDIGIT) SP *%x20-ff LF) - - packfile = PKT-LINE("packfile" LF) - *PKT-LINE(%x01-03 *%x00-ff) - - acknowledgments section - * If the client determines that it is finished with negotiations by - sending a "done" line (thus requiring the server to send a packfile), - the acknowledgments sections MUST be omitted from the server's - response. - - * Always begins with the section header "acknowledgments" - - * The server will respond with "NAK" if none of the object ids sent - as have lines were common. - - * The server will respond with "ACK obj-id" for all of the - object ids sent as have lines which are common. - - * A response cannot have both "ACK" lines as well as a "NAK" - line. - - * The server will respond with a "ready" line indicating that - the server has found an acceptable common base and is ready to - make and send a packfile (which will be found in the packfile - section of the same response) - - * If the server has found a suitable cut point and has decided - to send a "ready" line, then the server can decide to (as an - optimization) omit any "ACK" lines it would have sent during - its response. This is because the server will have already - determined the objects it plans to send to the client and no - further negotiation is needed. - - shallow-info section - * If the client has requested a shallow fetch/clone, a shallow - client requests a fetch or the server is shallow then the - server's response may include a shallow-info section. The - shallow-info section will be included if (due to one of the - above conditions) the server needs to inform the client of any - shallow boundaries or adjustments to the clients already - existing shallow boundaries. - - * Always begins with the section header "shallow-info" - - * If a positive depth is requested, the server will compute the - set of commits which are no deeper than the desired depth. - - * The server sends a "shallow obj-id" line for each commit whose - parents will not be sent in the following packfile. - - * The server sends an "unshallow obj-id" line for each commit - which the client has indicated is shallow, but is no longer - shallow as a result of the fetch (due to its parents being - sent in the following packfile). - - * The server MUST NOT send any "unshallow" lines for anything - which the client has not indicated was shallow as a part of - its request. - - wanted-refs section - * This section is only included if the client has requested a - ref using a 'want-ref' line and if a packfile section is also - included in the response. - - * Always begins with the section header "wanted-refs". - - * The server will send a ref listing ("<oid> <refname>") for - each reference requested using 'want-ref' lines. - - * The server MUST NOT send any refs which were not requested - using 'want-ref' lines. - - packfile-uris section - * This section is only included if the client sent - 'packfile-uris' and the server has at least one such URI to - send. - - * Always begins with the section header "packfile-uris". - - * For each URI the server sends, it sends a hash of the pack's - contents (as output by git index-pack) followed by the URI. - - * The hashes are 40 hex characters long. When Git upgrades to a new - hash algorithm, this might need to be updated. (It should match - whatever index-pack outputs after "pack\t" or "keep\t". - - packfile section - * This section is only included if the client has sent 'want' - lines in its request and either requested that no more - negotiation be done by sending 'done' or if the server has - decided it has found a sufficient cut point to produce a - packfile. - - * Always begins with the section header "packfile" - - * The transmission of the packfile begins immediately after the - section header - - * The data transfer of the packfile is always multiplexed, using - the same semantics of the 'side-band-64k' capability from - protocol version 1. This means that each packet, during the - packfile data stream, is made up of a leading 4-byte pkt-line - length (typical of the pkt-line format), followed by a 1-byte - stream code, followed by the actual data. - - The stream code can be one of: - 1 - pack data - 2 - progress messages - 3 - fatal error message just before stream aborts - -server-option -~~~~~~~~~~~~~ - -If advertised, indicates that any number of server specific options can be -included in a request. This is done by sending each option as a -"server-option=<option>" capability line in the capability-list section of -a request. - -The provided options must not contain a NUL or LF character. - - object-format -~~~~~~~~~~~~~~~ - -The server can advertise the `object-format` capability with a value `X` (in the -form `object-format=X`) to notify the client that the server is able to deal -with objects using hash algorithm X. If not specified, the server is assumed to -only handle SHA-1. If the client would like to use a hash algorithm other than -SHA-1, it should specify its object-format string. - -session-id=<session id> -~~~~~~~~~~~~~~~~~~~~~~~ - -The server may advertise a session ID that can be used to identify this process -across multiple requests. The client may advertise its own session ID back to -the server as well. - -Session IDs should be unique to a given process. They must fit within a -packet-line, and must not contain non-printable or whitespace characters. The -current implementation uses trace2 session IDs (see -link:api-trace2.html[api-trace2] for details), but this may change and users of -the session ID should not rely on this fact. - -object-info -~~~~~~~~~~~ - -`object-info` is the command to retrieve information about one or more objects. -Its main purpose is to allow a client to make decisions based on this -information without having to fully fetch objects. Object size is the only -information that is currently supported. - -An `object-info` request takes the following arguments: - - size - Requests size information to be returned for each listed object id. - - oid <oid> - Indicates to the server an object which the client wants to obtain - information for. - -The response of `object-info` is a list of the requested object ids -and associated requested information, each separated by a single space. - - output = info flush-pkt - - info = PKT-LINE(attrs) LF) - *PKT-LINE(obj-info LF) - - attrs = attr | attrs SP attrs - - attr = "size" - - obj-info = obj-id SP obj-size |