Network Working Group TL Laugstol Internet-Draft Arktekk Expires: May 14, 2012 November 2011 The Apache Maven Repository REST Applications and Protocol Bindings draft-apache-maven-repository-rest-applications-01 Abstract This document describe an architecture for a set of applications can access Maven repositories. The architecture is defined within the constrains of REST. Each application describe a useful interaction between a server and a client and the architecture ties the applications together with common traits and constraints. Status of this Memo This document is an Internet-Draft and is NOT offered inaccordance with Section 10 of RFC 2026, and the author does not provide the IETF with any rights other than to publish as an Internet-Draft. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on May 14, 2012. Table of Contents 1. TODOs and Notes . . . . . . . . . . . . . . . . . . . . . . . 2 2. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 2.1. Goals: . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2.2. Implementation goals . . . . . . . . . . . . . . . . . . . 3 2.3. Areas explicitly not covered . . . . . . . . . . . . . . . 3 2.4. Deployment scenarios . . . . . . . . . . . . . . . . . . . 4 3. Specification . . . . . . . . . . . . . . . . . . . . . . . . 4 3.1. Media Types . . . . . . . . . . . . . . . . . . . . . . . 4 3.2. Operations . . . . . . . . . . . . . . . . . . . . . . . . 4 3.3. Uploading Artifacts . . . . . . . . . . . . . . . . . . . 5 3.4. Downloading Artifacts . . . . . . . . . . . . . . . . . . 5 3.4.1. Download a single artifact . . . . . . . . . . . . . . 5 3.4.2. Download a set of artifacts . . . . . . . . . . . . . 5 3.5. Searching . . . . . . . . . . . . . . . . . . . . . . . . 5 3.6. List Repositories . . . . . . . . . . . . . . . . . . . . 6 4. Security Aspects . . . . . . . . . . . . . . . . . . . . . . . 6 4.1. Authentication and Authorization . . . . . . . . . . . . . 6 Laugstol (Category) [Page 1] Internet-Draft Apache Maven Repository REST Applications November 2011 5. User-Agent Considerations . . . . . . . . . . . . . . . . . . 6 5.1. Caching . . . . . . . . . . . . . . . . . . . . . . . . . 6 5.2. Multiple links . . . . . . . . . . . . . . . . . . . . . . 6 5.3. Forms . . . . . . . . . . . . . . . . . . . . . . . . . . 6 5.4. Server Responses . . . . . . . . . . . . . . . . . . . . . 6 5.4.1. Single Artifact . . . . . . . . . . . . . . . . . . . 6 5.4.2. Multiple Artifacts . . . . . . . . . . . . . . . . . . 6 5.4.2.1. text/uri-list . . . . . . . . . . . . . . . . . . 7 5.5. HTTP Binding . . . . . . . . . . . . . . . . . . . . . . . 7 6. Appendix: Words and Definitions . . . . . . . . . . . . . . . 7 7. Security Considerations . . . . . . . . . . . . . . . . . . . 7 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 7 8.1. Link Relation: http://maven.apache.org/rel/pom . . . . . . 8 8.2. Link Relation: http://maven.apache.org/rel/md5 . . . . . . 8 8.3. Link Relation: http://maven.apache.org/rel/sha1 . . . . . 8 8.4. Media Type: text/vnd.maven.pom . . . . . . . . . . . . . . 8 8.5. Media Type: text/vnd.maven.md5 . . . . . . . . . . . . . . 8 8.6. Media Type: text/vnd.maven.sha1 . . . . . . . . . . . . . 8 8.7. URN Registration: urn:maven:artifact . . . . . . . . . . . 8 9. Security Considerations . . . . . . . . . . . . . . . . . . . 8 10. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 8 11. References . . . . . . . . . . . . . . . . . . . . . . . . . . 8 11.1. Normative References . . . . . . . . . . . . . . . . . . 8 11.2. Informative References . . . . . . . . . . . . . . . . . 9 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 9 1. TODOs and Notes o How should an artifact be identified? * A "Maven Coordinate" HTTP header? Maven-Coordinate: groupId=com.example, artifactId=example, type=jar, classifiers=awesome, version=1.2.3 * A maven URI type? maven://groupId/artifactId/version? * A maven URN prefix? urn:maven:artifact:groupId=..:artifactId=. .:classifier=..:type=..:version=.. o Promising media types and link relations: * "describedby" link relation - a link from an artifact to its POM * "edit" link relation describing an external form of editing. useful if we don't want to include all the edit controls inside feeds. Reduces bandwidth and latency * atom and collection+JSON seem like nice formats for manipulating lists of artifacts. * metalink looks like a nice way to include Laugstol (Category) [Page 2] Internet-Draft Apache Maven Repository REST Applications November 2011 + There are both a standalone XML format and a HTTP binding that can be used. It might be feasible to include metalink inside atom and possibly metalink inside collection+JSON Put this where it'll fit later: It is assumed that there will always only be a single instance of an artifact + version. If two repositories can resolve the same artifact any user agent can choose arbitrary which one it'll like to use. This is important as some people download an artifact from one repository, modify it and upload it to another repository. This is sometimes done by operations when they consume artifacts from a "developer" repository, inject some configuration variables and pushes it to a "production" repository. 2. Introduction The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119]. 2.1. Goals: o Any user agent should be able to rely on caching information given by the repository. No explicit caching should be necessary by the application using a user agent for a Maven repository. In other words, no ~/.m2/repository or ~/.ivy2. o It should be possible to use standard HTTP intermediates without interfering. This is required to cover several use cases: * Teams inside a corporation might be far away from a central repository and/or have a slow link. A local cache will be able to answer their queries faster. * A central repository might get a very high number of requests. A cache next to the back-end application will help scale the repository as it will take load of the back-end application. o The applications should be easy to use with current command line clients. 2.2. Implementation goals o There shouldn't be any need to invent lots of media types or link relations. The operations should be able to fit within existing hyper medias. 2.3. Areas explicitly not covered Laugstol (Category) [Page 3] Internet-Draft Apache Maven Repository REST Applications November 2011 This specification will not apply to the existing static Maven repositories.That would be impossible within reason and there are already enough tools to support that kind of repositories. However, it inside this specification to be able to create an application that support the operations while still points to a static repository for providing the actual artifacts. 2.4. Deployment scenarios o Standalone repository - similar to nexus, Artifactory etc o Mirror application * rsync-ed copy of another repository with extra meta data * Partial mirror of another repository o Security proxy of another mirror * can require authentication 3. Specification 3.1. Media Types TODO: Find a reference to @mamund's "you should always design a new media type (but not format)". 3.2. Operations o Upload artifact * Single files * Single artifact + signatures and checksums? o Upload set of artifacts - zipped? multipart? o Move/copy artifacts between repositories * promote artifacts from staging to test to production * copy from internal repository to external repository o Sign artifact o Lists of recently uploaded/changed files o Download bigger chunks of an archive * Download entire applications with one GET Laugstol (Category) [Page 4] Internet-Draft Apache Maven Repository REST Applications November 2011 * Download the latest changes from a repository o Searching * Search for group id, artifact id, by type, by classifier, signed by * Search content of artifacts - find all jars with class foo.Bar, all classes that implement javax.servlet.Servlet 3.3. Uploading Artifacts It should be possible to: o Upload a single artifact o Upload a bundle of artifacts in a single upload o Attach signatures to an artifact 3.4. Downloading Artifacts 3.4.1. Download a single artifact Steps to download a single artifact with a specific version: Download the start resource and find a form named "download- artifact". Artifacts can be identified either through an artifact URN or a full set of coordinates. To use the URN mechanism fill out the "urn" field, to use the coordinate version fill out the fields named "groupId", "artifactId", "version", "type" and "classifier". The "classifier" field is optional. These fields MUST be present in the form. TODO: Some possible optional controls: "filename". Submit the form. The server MAY return a single artifact, see Section 5.4.1. 3.4.2. Download a set of artifacts The set might be a search, or just a list of artifacts. The server is free to choose to send a container format with all the files in it, or just an URL list with pointers. 3.5. Searching Servers SHOULD implement conditional controls so it's possible to poll the server regularly. TODO: specify how to query for only changes after a certain timestamp. Laugstol (Category) [Page 5] Internet-Draft Apache Maven Repository REST Applications November 2011 3.6. List Repositories 4. Security Aspects 4.1. Authentication and Authorization Authentication and authorization is generally not specified by this specification as it is assumed that the transport mechanisms will specify the different options for authentication. 5. User-Agent Considerations 5.1. Caching A user-agent should include information on how stale data it allows when requesting artifacts. This make it possible for caches to include caching information in the return response that intermediates also will use. TODO: Are there any standard ways in HTTP to do this? A user agent must always check that it got the data it required after receiving a response. This means checking response code and content type in particular. 5.2. Multiple links If a response include many links with the same relation and content type, it should can try all links to get the data. This makes it possible for a server to include pointers to many servers that can respond to the same request. A user agent should try to stick to the same host (as defined by the URI) for the following requests. TODO: It this mainly covered by metalink? 5.3. Forms When the user agent is downloading a form with unknown controls they MUST be sent back to the server with any value specified. 5.4. Server Responses 5.4.1. Single Artifact HTTP responses SHOULD include a "Content-Disposition" header with Disposition Type set to "attachment" to indicates that the artifact shouldn't be processed according to its media type. See also Content-Disposition in HTTP [RFC6266]. Those implementations that use generic HTTP servers to serve the artifacts might not include the header as it could be shared between many applications. 5.4.2. Multiple Artifacts Laugstol (Category) [Page 6] Internet-Draft Apache Maven Repository REST Applications November 2011 When the server replies with a set of artifacts it MAY send one of the following responses. If the response is one of the specified responses, it MUST follow this specification. 5.4.2.1. text/uri-list See the text/uri-list specification [RFC2483]. TODO: Specify "application/atom+xml", "text/collection+json" responses too. 5.5. HTTP Binding Explain what extra headers can be given with each response for artifacts: o Checksums: MD5, SHA-1 o Cryptographic signatures, either complete signatures or links to signatures o File size 6. Appendix: Words and Definitions Maven Repository A Maven Repository contain a set of artifacts. Each artifact is identified by a: * group id * artifact id * version * type * classifier - this is optional See Also (The Maven site at apache)[http://maven.apache.org] 7. Security Considerations 8. IANA Considerations o TODO: Move media types here or this section down. o TODO: Check if there are other checksum relations that are as useful. It might be possible to reuse something here. Would be make sense to register a "text/checksum" type and assume that all clients can use content negotiation to find it? Should it be registered at all, or should it be assumed that the application always can implement RFC3230? Laugstol (Category) [Page 7] Internet-Draft Apache Maven Repository REST Applications November 2011 8.1. Link Relation: http://maven.apache.org/rel/pom A pointer to a Maven POM file. TODO: Should probably not be here, a link with rel=describedby, contentType=text/vnd.maven.pom is probably fine. 8.2. Link Relation: http://maven.apache.org/rel/md5 8.3. Link Relation: http://maven.apache.org/rel/sha1 8.4. Media Type: text/vnd.maven.pom 8.5. Media Type: text/vnd.maven.md5 8.6. Media Type: text/vnd.maven.sha1 8.7. URN Registration: urn:maven:artifact Should it be "urn:maven:artifact" or "urn:maven" Use template in http://tools.ietf.org/html/rfc3406#appendix-A Creating an informal registration seems the right way to do as long as this document doesn't become an RFC. 9. Security Considerations 10. Acknowledgments 11. References 11.1. Normative References [RFC1864] Myers, J. and M. Rose, "The Content-MD5 Header Field", RFC 1864, October 1995. [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [RFC2483] Mealling, M. and R. Daniel, "URI Resolution Services Necessary for URN Resolution", RFC 2483, January 1999. [RFC2616] Fielding, R., Gettys, J., Mogul, J., Frystyk, H., Masinter, L., Leach, P. and T. Berners-Lee, "Hypertext Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999. [RFC3230] Mogul, J. and A. Van Hoff, "Instance Digests in HTTP", RFC 3230, January 2002. [RFC4287] Nottingham, M.Ed., and R. Sayre, Ed., "The Atom Syndication Format", RFC 4287, December 2005. Laugstol (Category) [Page 8] Internet-Draft Apache Maven Repository REST Applications November 2011 [RFC4627] Crockford, D., "The application/json Media Type for JavaScript Object Notation (JSON)", RFC 4627, July 2006. [RFC5854] Bryan, A., Tsujikawa, T., McNab, N. and P. Poeml, "The Metalink Download Description Format", RFC 5854, June 2010. [RFC6249] Bryan, A., McNab, N., Tsujikawa, T., Poeml, P. and H. Nordstrom, "Metalink/HTTP: Mirrors and Hashes", RFC 6249, June 2011. [RFC6266] Reschke, J., "Use of the Content-Disposition Header Field in the Hypertext Transfer Protocol (HTTP)", RFC 6266, June 2011. 11.2. Informative References [RFC1321] Rivest, R., "The MD5 Message-Digest Algorithm", RFC 1321, April 1992. [RFC3174] Eastlake, D. and P. Jones, "US Secure Hash Algorithm 1 (SHA1)", RFC 3174, September 2001. [RFC3406] Daigle, L., van Gulik, D., Iannella, R. and P. Faltstrom, "Uniform Resource Names (URN) Namespace Definition Mechanisms", BCP 66, RFC 3406, October 2002. Author's Address Trygve Arktekk NO Phone: +47 926 99 070 Email: trygvis@inamo.no Laugstol (Category) [Page 9]