REST - Part 11 Statelessness

As per the REST (REpresentational “State” Transfer) architecture, the server does not store any state about the client session on the server side. This restriction is called Statelessness. Each request from the client to server must contain all of the information necessary to understand the request, and cannot take advantage of any stored context on the server. Session state is therefore kept entirely on the client. client is responsible for storing and handling all application state related information on client side.

It also means that the client is responsible for sending any state information to the server whenever it is needed. There should not be any session affinity or sticky sessions on server.

Statelessness means that every HTTP request happens in complete isolation. When the client makes an HTTP request, it includes all information necessary for the server to fulfill that request. The server never relies on information from previous requests. If that information was important, the client would have sent it again in this request.
To enable clients to access these stateless APIs, its necessary that servers also should include every piece of information that the client may need to create the state on its side.

For becoming stateless, do not store even authentication/authorization details of client. Provide credentials with the request. Each request MUST stand alone and should not be affected by the previous conversation happened from the same client in past.

Application State vs Resource State
Please do not confuse between application state and resource state. Both are completely different things.

Application state is server-side data which servers store to identify incoming client requests, their previous interaction details, and current context information.

Resource state is the current state of a resource on a server at any point of time – and it has nothing to do with the interaction between client and server. It is what you get as a response from the server as API response. You refer to it as resource representation.

REST statelessness means being free on application state.

Advantages of Statelessness
There are some very noticeable advantages for having REST APIs stateless.

Statelessness helps in scaling the APIs to millions of concurrent users by deploying it to multiple servers. Any server can handle any request because there is no session related dependency.
Being stateless makes REST APIs less complex – by removing all server-side state synchronization logic.
A stateless API is also easy to cache as well. A specific software can decide whether or not to cache the result of an HTTP request just by looking at that one request. There’s no nagging uncertainty that state from a previous request might affect the cacheability of this one. It improves the performance of applications.
The server never loses track of “where” each client is in the application because the client sends all necessary information with each request.

REST - Part 10 REST API Versioning

To manage this complexity, version your API. Versioning helps you iterate faster when the needed changes are identified.

Change in an API is inevitable as your knowledge and experience of a system improve. Managing the impact of this change can be quite a challenge when it threatens to break existing client integration.
When to Version
APIs only need to be up-versioned when a breaking change is made. Breaking changes include:

a change in the format of the response data for one or more calls
a change in the response type (i.e. changing an integer to a float)
removing any part of the API.
Breaking changes should always result in a change to the major version number for an API or content response type.

Non-breaking changes, such as adding new endpoints or new response parameters, do not require a change to the major version number. However, it can be helpful to track the minor versions of APIs when changes are made to support customers who may be receiving cached versions of data or may be experiencing other API issues.

How to Version
REST doesn’t provide for any specific versioning guidelines but the more commonly used approaches fall into three categories:

URI Versioning
Using the URI is the most straightforward approach (and most commonly used as well) though it does violate the principle that a URI should refer to a unique resource. You are also guaranteed to break client integration when a version is updated.

e.g.

http://api.example.com/v1
http://apiv1.example.com
The version need not be numeric, nor specified using the “v[x]” syntax. Alternatives include dates, project names, seasons or other identifiers that are meaningful enough to the team producing the APIs and flexible enough to change as the versions change.

Versioning using Custom Request Header
A custom header (e.g. Accept-version) allows you to preserve your URIs between versions though it is effectively a duplicate of the content negotiation behavior implemented by the existing Accept header.

e.g.

Accept-version: v1
Accept-version: v2
Versioning using Accept header
Content negotiation may let you preserve a clean set of URLs but you still have to deal with the complexity of serving different versions of content somewhere. This burden tends to be moved up the stack to your API controllers which become responsible for figuring out which version of a resource to send. The end result tends to be a more complex API as clients have to know which headers to specify before requesting a resource.

e.g.

Accept: application/vnd.example.v1+json
Accept: application/vnd.example+json;version=1.0
In the real world, an API is never going to be completely stable. So it’s important how this change is managed. A well documented and gradual deprecation of API can be an acceptable practice for most of the APIs.

REST - Part 9 REST API Security Essentials

REST API Security Essentials
Security isn’t an afterthought. It has to be an integral part of any development project and also for REST APIs. There are multiple ways to secure a RESTful API e.g. basic auth, OAuth etc. but one thing is sure that RESTful APIs should be stateless – so request authentication/authorization should not depend on cookies or sessions. Instead, each API request should come with some sort authentication credentials which must be validated on the server for each and every request.

REST Security Design Principles
The paper “The Protection of Information in Computer Systems” by Jerome Saltzer and Michael Schroeder, put forth eight design principles for securing information in computer systems, as described in the following sections:

Least Privilege: An entity should only have the required set of permissions to perform the actions for which they are authorized, and no more. Permissions can be added as needed and should be revoked when no longer in use.
Fail-Safe Defaults: A user’s default access level to any resource in the system should be “denied” unless they’ve been granted a “permit” explicitly.
Economy of Mechanism: The design should be as simple as possible. All the component interfaces and the interactions between them should be simple enough to understand.
Complete Mediation: A system should validate access rights to all its resources to ensure that they’re allowed and should not rely on cached permission matrix. If the access level to a given resource is being revoked, but that isn’t reflected in the permission matrix, it would violate the security.
Open Design: This principle highlights the importance of building a system in an open manner—with no secret, confidential algorithms.
Separation of Privilege: Granting permissions to an entity should not be purely based on a single condition, a combination of conditions based on the type of resource is a better idea.
Least Common Mechanism: It concerns the risk of sharing state among different components. If one can corrupt the shared state, it can then corrupt all the other components that depend on it.
Psychological Acceptability: It states that security mechanisms should not make the resource more difficult to access than if the security mechanisms were not present. In short, security should not make worse the user experience.
Best Practices to Secure REST APIs
Below given points may serve as a checklist for designing the security mechanism for REST APIs.

Keep it Simple
Secure an API/System – just how secure it needs to be. Every time you make the solution more complex “unnecessarily”, you are also likely to leave a hole.

Always Use HTTPS
By always using SSL, the authentication credentials can be simplified to a randomly generated access token that is delivered in the username field of HTTP Basic Auth. It’s relatively simple to use, and you get a lot of security features for free.

If you use HTTP 2, to improve performance – you can even send multiple requests over a single connection, that way you avoid the complete TCP and SSL handshake overhead on later requests.

Use Password Hash
Passwords must always be hashed to protect the system (or minimize the damage) even if it is compromised in some hacking attempt. There are many such hashing algorithms which can prove really effective for password security e.g. MD5, SHA, PBKDF2, bcrypt and scrypt algorithms.

Never expose information on URLs
Usernames, passwords, session tokens, and API keys should not appear in the URL, as this can be captured in web server logs, which makes them easily exploitable.

https://api.domain.com/user-management/users/{id}/someAction?apiKey=abcd123456789  //Very BAD !!
Above URL exposes API key So, never use this form of security.

Consider OAuth
Though basic auth is good enough for most of the APIs and if implemented correctly, it’s secure as well – yet you may want to consider OAuth as well. The OAuth 2.0 authorization framework enables a third-party application to obtain limited access to an HTTP service, either on behalf of a resource owner by orchestrating an approval interaction between the resource owner and the HTTP service, or by allowing the third-party application to obtain access on its own behalf.

Consider Adding Timestamp in Request
Along with other request parameters, you may add a request timestamp as HTTP custom header in API request. The server will compare the current timestamp to the request timestamp, and only accepts the request if it is within a reasonable timeframe (1-2 minutes, perhaps).

This will prevent very basic replay attacks from people who are trying to brute force your system without changing this timestamp.

Input Parameter Validation
Validate request parameters on the very first step, before it reaches to application logic. Put strong validation checks and reject the request immediately if validation fails. In API response, send relevant error messages and example of correct input format to improve user experience.

REST - Part 8 Idempotent REST APIs

In the context of REST APIs, when making multiple identical requests has the same effect as making a single request – then that REST API is called idempotent.

When you design REST APIs, you must realize that API consumers can make mistakes. They can write client code in such a way that there can be duplicate requests as well. These duplicate requests may be unintentional as well as intentional some time (e.g. due to timeout or network issues). You have to design fault-tolerant APIs in such a way that duplicate requests do not leave the system unstable.

An idempotent HTTP method is an HTTP method that can be called many times without different outcomes. It would not matter if the method is called only once, or ten times over. The result should be the same. It essentially means that the result of a successfully performed request is independent of the number of times it is executed. For example, in arithmetic, adding zero to a number is idempotent operation.
If you follow REST principles in designing API, you will have automatically idempotent REST APIs for GET, PUT, DELETE, HEAD, OPTIONS and TRACE HTTP methods. Only POST APIs will not be idempotent.

POST is NOT idempotent.
GET, PUT, DELETE, HEAD, OPTIONS and TRACE are idempotent.
Let’s analyze how above HTTP methods end up being idempotent – any why POST is not.

HTTP POST
Generally – not necessarily – POST APIs are used to create a new resource on server. So when you invoke the same POST request N times, you will have N new resources on the server. So, POST is not idempotent.

HTTP GET, HEAD, OPTIONS and TRACE
GET, HEAD, OPTIONS and TRACE methods NEVER change the resource state on server. They are purely for retrieving the resource representation or meta data at that point of time. So invoking multiple requests will not have any write operation on server, so GET, HEAD, OPTIONS and TRACE are idempotent.

HTTP PUT
Generally – not necessarily – PUT APIs are used to update the resource state. If you invoke a PUT API N times, the very first request will update the resource; then rest N-1 requests will just overwrite the same resource state again and again – effectively not changing anything. Hence, PUT is idempotent.

HTTP DELETE
When you invoke N similar DELETE requests, first request will delete the resource and response will be 200 (OK) or 204 (No Content). Other N-1 requests will return 404 (Not Found). Clearly, the response is different from first request, but there is no change of state for any resource on server side because original resource is already deleted. So, DELETE is idempotent.

Please keep in mind if some systems may have DELETE APIs like this:

DELETE /item/last
In the above case, calling operation N times will delete N resources – hence DELETE is not idempotent in this case. In this case, a good suggestion might be to change above API to POST – because POST is not idempotent.

POST /item/last
Now, this is closer to HTTP spec – hence more REST compliant.

REST - Part 7 HATEOAS Driven REST APIs

HATEOAS (Hypermedia as the Engine of Application State) is a constraint of the REST application architecture that keeps the RESTful style architecture unique from most other network application architectures. The term “hypermedia” refers to any content that contains links to other forms of media such as images, movies, and text.

This architectural style lets you use hypermedia links in the response contents so that the client can dynamically navigate to the appropriate resource by traversing the hypermedia links. This is conceptually the same as a web user navigating through web pages by clicking the appropriate hyperlinks in order to achieve a final goal.

For example, below given JSON response may be from an API like HTTP GET http://api.domain.com/management/departments/10

{
    "departmentId": 10,
    "departmentName": "Administration",
    "locationId": 1700,
    "managerId": 200,
    "links": [
        {
            "href": "10/employees",
            "rel": "employees",
            "type" : "GET"
        }
    ]
}
In the preceding example, the response returned by the server contains hypermedia links to employee resources 10/employees, which can be traversed by the client to read employees belonging to the department.

The advantage of the above approach is that hypermedia links returned from the server drive the application’s state and not the other way around.

There is no universally accepted format for representing links between two resources in JSON. You may choose to adopt the above format or you may choose to send links in HTTP response headers.

HTTP/1.1 200 OK
...
Link: <10/employees>; rel="employees"
Both are absolutely valid solutions.

HATEOAS Implementation
In the real world, when you visit a website – you hit its homepage. It presents some snapshots and links to other sections of websites. You click on them and then you get more information along with more related links which are relevant to the context.

Similar to a human’s interaction with a website, a REST client hits an initial API URI and uses the server-provided links to dynamically discover available actions and access the resources it needs. The client need not have prior knowledge of the service or the different steps involved in a workflow. Additionally, the clients no longer have to hard code the URI structures for different resources. This allows the server to make URI changes as the API evolves without breaking the clients.

Above API interaction is possible using HATEOAS only.

Each REST framework provides it’s own way on creating the HATEOAS links using framework capabilities e.g. in this RESTEasy HATEOAS tutorial, links are part of resource model classes which is transferred as resource state to the client.

HATEOAS References
The following are the two popular formats for specifying JSON REST API hypermedia links:

RFC 5988 (web linking)
RFC 5988 puts forward a framework for building links that defines the relation between resources on the web. Each link in RFC 5988 contains the following properties:

Target URI: Each link should contain a target Internationalized Resource Identifiers (IRIs). This is represented by the href attribute.

Link relation type: The link relation type describes how the current context (source) is related to the target resource. This is represented by the rel attribute.

Attributes for target IRI: The attributes for a link include hreflang, media, title, and type, and any extension link parameters.

JSON Hypermedia API Language (HAL)
JSON HAL is a promising proposal that sets the conventions for expressing hypermedia controls, such as links, with JSON or XML. It is in the draft stage at this time.

The two associated MIME types are

media type: application/hal+xml
media type: application/hal+json
Each link in HAL may contain the following properties:

Target URI: It indicates the target resource URI. This is represented by the href attribute.
Link relation: The link relation type describes how the current context is related to the target resource. This is represented by the rel attribute.
Type: This indicates the expected resource media type. This is represented by the type attribute.

There is no right or wrong in choosing a hypermedia link format for your application. You should pick up a format that meets most of your use case requirements and stick to it.

REST - Part 6 Caching REST API Response

Generally, resources can have multiple presentations, mostly because there may be multiple different clients expecting different representations. Asking for a suitable presentation by a client, is referred as content negotiation.

HTTP has provisions for several mechanisms for “content negotiation” — the process of selecting the best representation for a given response when there are multiple representations available.
RFC 2616
Server-driven Vs Agent-driven Content Negotiation
If the selection of the best representation for a response is made by an algorithm located at the server, it is called server-driven negotiation. If that selection is made at agent or client side, its called agent-driven content negotiation.

Practically, you will NOT find much usage of server side negotiations because in that way, you have to make lots of assumptions about client expectations. Few things like client context or how client will use the resource representation is almost impossible to determine. Apart from that this approach makes the server side code complex, unnecessarily.

So, most REST API implementations rely on agent driven content negotiations. Agent driven content negotiation rely on usage of HTTP request headers or resource URI patterns.

Content negotiation using HTTP headers
At server side, an incoming request may have an entity attached to it. To determine it’s type, server uses the HTTP request header Content-Type. Some common examples of content types are “text/plain”, “application/xml”, “text/html”, “application/json”, “image/gif”, and “image/jpeg”.

Content-Type: application/json
Similarly, to determine what type of representation is desired at client side, HTTP header ACCEPT is used. It will have one of the values as mentioned for Content-Type above.

Accept: application/json
Generally, if no Accept header is present in the request, the server can send pre-configured default representation type.

Implementing Accept header based content negotiation is most used and recommened way.
Content negotiation using URL patterns
Another way to pass content type information to server, client may use specific extension in resource URIs. For example, a client can ask for details using:

http://rest.api.com/v1/employees/20423.xml
http://rest.api.com/v1/employees/20423.json
In above case, first request URI will return a XML response whether second request URI will return a JSON response.

Defining preferences
If is possible to have multiple values in Accept header. Client may want to give multiple values in accept header when client is not sure about if its desired representation is present or supported by server at that time. [RFC 2296]

For example,

Accept: application/json,application/xml;q=0.9,*/*;q=0.8

Above Accept header allows you to ask the server a JSON format. If it can’t, perhaps it could return XML format (the second level). If it’s still not possible, let it return what it can.

The preference order is defined through the q parameter with values from 0 to 1. When nothing is specified, the implicit value is 1.

REST - Part 5 Caching REST API Response

REST APIs can return the resource representations in a number of formats such as XML, JSON, HTML or even plain text. All such forms can be compressed to a lesser number of bytes to save bandwidth over the network. Different protocols use different techniques to enable compression and notify the clients about compression scheme – so that client can decompress it before consuming the representations.

Compression, like encryption, is something that happens to a representation in transit and must be undone before the client can use the representation.
HTTP is most widely used protocol for REST – so I am taking example of HTTP specific response compression.

Compression Related Request/Response Headers
Accept-Encoding
While requesting resource representations – along with an HTTP request the client sends an Accept-Encoding header that says what kind of compression algorithms the client understands.

The two standard values for Accept-Encoding are compress and gzip.

A sample request with accept-encoding header looks like this :

GET        /employees         HTTP/1.1
Host:     www.domain.com
Accept:     text/html
Accept-Encoding:     gzip,compress
Other possible usage of accept-encoding may be:

Accept-Encoding: compress, gzip
Accept-Encoding:
Accept-Encoding: *
Accept-Encoding: compress;q=0.5, gzip;q=1.0
Accept-Encoding: gzip;q=1.0, identity; q=0.5, *;q=0
If an Accept-Encoding field is present in a request, and if the server cannot send a response which is acceptable according to the Accept-Encoding header, then the server SHOULD send an error response with the 406 (Not Acceptable) status code.

Content-Encoding
If the server understands one of the compression algorithms from Accept-Encoding, it can use that algorithm to compress the representation before serving it. When successfully compressed, server lets know the client of encoding scheme by another HTTP header i.e. Content-Encoding.

200 OK
Content-Type:     text/html
Content-Encoding:     gzip
If the content-coding of an entity in a request message is not acceptable to the origin server, the server SHOULD respond with a status code of 415 (Unsupported Media Type). If multiple encodings have been applied to an entity, the content encodings MUST be listed in the order in which they were applied.

Please note that original media-type for request and response are not impacted whether compression is requested or not.

Compression can save a lot of bandwidth, with very little cost in additional complexity. Also, you may know that most web browsers automatically request compressed representations from website host servers – using above headers.