More About HTTP Protocol

Tutorial Introduction

Hi again and welcome in the 2nd part of “Building N-Tier RESTful API using ASP.NET Web API 2” Tutorial.

This part indeed intended to be part of the tutorial but it’s totally independent of ASP.NET as i’ll be discussing HTTP Protocol in abstract so if you’re interested to know more about the HTTP Protocol and you’re not a .NET Developer it’s still the right place for you ^^

It has been a long time since i wrote the first part and i didn’t continue to deliver the tutorial content i’m sorry for that, Now i promise that i’ll continue to deliver the whole content of the tutorial as i’ll write a new part on weekly basis, Hope you’ll enjoy.

In this tutorial you’ll learn how to develop an API using ASP.NET Web API 2, N-Tier Architecture, and a lot of stuff that will help you to get started building your awesome API confidently

Tutorial Parts

  1. Build Your First API using Web API 2
  2. More About HTTP Protocol

Tutorial Pre-Requests

  • Strong knowledge of OOP and C#
  • Experience with Web Development
  • Experience with JavaScript
  • A little knowledge of AngularJS (To build client app at the end of the tutorial)

What you’ll learn in this tutorial

I’m actually planning a lot of stuff and want to make it a complete tutorial so

  • RESTful API architecture
  • ASP.NET Web API 2
  • N-Tier architecture
  • Dependency Injection
  • OData protocol
  • Build a client application with AngularJS and awesome Angular Material for the UI
  • And a lot more..

Source Code:

You can find the code for this tutorial in this GitHub Repository.

Part 2: More About HTTP Protocol

HTTP Protocol is the heart of the web nowadays so it’s very important for web developers to have a great understanding of what’s the HTTP Protocol.

HTTP stands for “Hypertext Transfer Protocol”, Great so what’s that means? Lets split this definition into 3 segements

  • Hypertext: Is a text that contains links to another text. Given an example the blog post’s page you’re reading right now. It’s a text that contains links or references to another text you can click the link to get navigated to the other text.
  • Transfer: At some point the text you’re reading is located somewhere on the earth in some storage device. In order for you to read this text it has to be transferred to your device and that what’s transfer refers to. So using network communication between your device and wordpress server the text of this blog is being transferred to your device.
  • Protocol: Some guys are getting worried about this word but it isn’t this hard to understand. Okay assume you’re working in a warehouse and I’ll ship some products to you. So you’ll ask me for some stuff and I’ll ship it safely to your warehouse. But in order for this to work perfectly there has to be some agreement that we both know to minimize the mistakes for example there has to be a document describing the items being shipped a long with its quantities, ids, price, ..Etc. Also the items should arrive in closed boxes that’s labeled with the item id and quantity. If we didn’t do this every time I’ll deliver you the order and you’ll do millions of mistakes because you don’t know what’s product in which box and how many quantities. And that’s all should results a lot of mistakes. Simply this is called a Protocol.

So the HTTP is simply the agreement between all network devices using this protocol to understand the way in which the hypertext is going to be transferred via a network communication. HTTP is the heart of the internet nowadays everyone is using HTTP daily surfing the internet.

HTTP is Stateless Protocol

Well stateless means that you’re going to ask me to buy a product and I’ll ask you for your ID then I’ll give it to you. If you left and came back even after a second to buy another thing I’ll ask you for your ID again (I don’t know you, I Forgot).

That means HTTP Protocol treats each request independently of any other requests, Unlike TCP protocol which is stateful. However it’s great to note that HTTP uses TCP for transportation. Here’s come a question how can HTTP be stateless while its relying on a stateful protocol TCP. To answer that let’s give an example of your cell phone. Let’s assume that your cell phone is establishing a stateful connection to the network provider using the TCP protocol. And i’ve decided to call you so i’ve sent a HTTP request to do that, And you’ve responded and we had a great call then we’ve disconnected. After we disconnect the call you’re not anymore with me on the phone, And if want to call you again it’s a totally new call that’s independentent of the previous one.

It’s all about Request/Response

HTTP Protocol is all about a client that sends an HTTP Request to the server and the server process the request and respond back with an HTTP Response.

Figure 1- HTTP Request/Response

Understanding HTTP Request

HTTP Request is the way of how client will communicate with the server via HTTP Protocol, As we previously mentioned what’s protocol so an HTTP Request is having a well-defined elements and should be used correctly in order to follow the protocol guidelines.

HTTP Request Elements

1- URL

Of course everyone knows what URLs is. Well given an example this url let’s split it into multiple segments

  1. http:// is the protocol that’s used it can be HTTPS instead of HTTP we may speak about HTTPS later but currently it’s out of the blog post scope
  2. is the Domain and it’s actually a unique name reserved for some server in the web. Your browser will look up for this domain in some DNS to get the server IP address that’s associated with this domain name in order to send the request to.
  3. /articles this is called the Resource Path and it’s handled by the server to locate the resource you’re looking for. Like in the previous part we’ve created some controllers and actions that we requested via its corresponding resource path.
  4. ?title=http&author=ibrahim This is called a Query String and we’ll check how it’s used later in this series. It’s a way of passing parameters to the server via the URL that it’ll going to be used somehow for purposes like filtering the results or it’ll be ignored by the server.

2- HTTP Methods (Verbs)

You want to send a request to the server and the server should take an action upon this request whether retrieving, creating, modifying or deleting some data. HTTP Methods describes the action that the server should take.

HTTP Methods are a well-defined methods by the standard Find it here (below a table describing most commonly used HTTP Methods)

Safe Methods: some HTTP methods are marked as “safe”. Which means they wont make any changes to a resource on the server and they’re used only for retrieval
Idempotent Methods: are methods that can be called many times without different outcome. (All safe methods are idempotent).

Method Description Safe Idempotent
GET Used to retrieve a resource Yes Yes
HEAD Used to obtain meta-information about the resource (HEAD is identical to get but doesn’t have request body, HEAD response header should be identical as if it was GET) Yes Yes
POST Used to create a new resource on the server No No
PUT Used to modify a resource (or create new one if not exists) No Yes
PATCH Used for a partial resource modifications No No
DELETE Used to delete a resource on the server. No Yes
OPTIONS Usually used before any preflight cross-origin request in order to know whether it’s safe to do it. Yes Yes

3- Request Headers:

Request headers are used to carry some information (metadata) about the request itself. Like Authentication Token, The language you want to retrieve the data with, ..etc. We’ll use many HTTP Headers in practice but not in this part.

HTTP Headers are well-defined headers by the standard some of it is used to be sent with the request and some other to be returned along with the response Find it here, However you can create your own Custom Header that you know how to use it on the server. (below is a list of the most commonly used Request Headers)

Header Usage
Accept What data formats in which the client wants. (e.g., text/html, application/xml)
Accept-Language What languages in whic the client wants the data to be in. (e.g., en-US, ar-EG)
Accept-Charset The characterset in which the client wants. (e.g., utf8, ISO-8859-1)
Cookie Used to send the cookie that’s saved on the user’s browser that could contain some information about the user to the server.
Authorization Used to send authorization token to verify if the requester is authenticated

There’s a lot more headers we were just peeking some of them. We’ll get in depth with HTTP Headers later on in the tutorial.

4- Request Body:

Request body is used to send data to the server. Calling from the previous part we wanted to create a news article so we sent the data in which we want the article to be created for the server to process.

The server is responsible for validating the request body’s data and if it matches the structure and contains the mandatory data for the resource to be created/modified, hence the server should process the request and return a success indicator otherwise the server should return an error

Here’s a figure showing request elements and how it look likes. I’ve used fiddler to capture HTTP Requests it’s a very useful tool that’ll going to need it Find it here

Figure 2- Showing Request Elements

Understanding HTTP Response

Okay a client has sent a request to the server. Now it’s the server turn to check the request elements to do the corresponding action, After the server finished its work it’ll respond back with an HTTP Response

Like HTTP Request an HTTP Response have some elements that describes what’s it all about.

HTTP Response Elements

1- Status Code

Assuming you’ve taken an exam in college and the result is now published. What’s the first thing you’ll look for is it the reasons and the grade of each question or what’s the answers you failed in.

Of course not you’ll look for some Indicator of Success and it’s likely a well defined range values [Excellent, Very Good, Good, Passed, Failed]

Same here the HTTP Response comes along with a Status Code that should be used to indicate the success of the request. The standard defines wide range of codes to indicate the action result as well not only success or failed Find it here (below a list of most commonly used status codes)

  • 100 Continue: Means server has recieved the request headers and the client should proceed and send the body.
  • 101 Switching Protocol: Means that client asked for switching the protocol and the server accepted.
  • 102 Processing: Means that server has recieved and is processing the request but no response available yet. (If the processing takes to long so this’s sent to prevent the client from timing out and assume that the request was lost)
  • 103 Checkpoint: Used in resumable requests proposal to resume aborted POST, PUT (think of it as an aborted upload scenario)

2xx Success:
Indicates that the action requested has been completed successfully

  • 200 Ok: The standard response for successful HTTP Requests. Response has a body enclosing a representation of the resource requested.
  • 201 Created: Indicates that a new resource has been created succesfully. Usually response has a body enclosing a representation of the newly created resource and a Location Header with a url to ‘GET’ the created resource.
  • 202 Accepted: The request has been accepted but the processing has not been completed.
  • 203 Non-Authoritative Information: The request has been processed successfully but is returning information that maybe from another source
  • 204 No-Content: The request has been processed successfully. But the server not returning any content (Empty Body).
  • 205 Reset-Content: Just like (204 No-Content). But requires the requestor to reset the document view (for instance clear the form for new input).
  • 206 Partial-Content: The server has processed the partial GET request (Usually used for retrieving big size data)

3xx Redirect:

  • 300 Multiple Choices: Server has several options and may choose one based on requestor (User Agent) or will give a list of options to the requestor to choose form.
  • 301 Moved Permanently: The requested page has been permanently moved to another location (When server responds with this status code the client automatically moved to the new location).
  • 302 Found: This response code means that URI of requested resource has been changed temporarily. New changes in the URI might be made in the future. Therefore, this same URI should be used by the client in future requests.
  • 303 See Other: Server sent this response to directing client to get requested resource to another URI with an GET request.
  • 304 Not Modified: This is used for caching purposes. It is telling to client that response has not been modified. So, client can continue to use same cached version of response.
  • 305 Use Proxy: This means requested response must be accessed by a proxy. This response code is not largely supported because security reasons.
  • 306 Unused: This response code is no longer used, it is just reserved currently. It was used in a previous version of the HTTP 1.1 specification.
  • 307 Temporary Redirect: Server sent this response to directing client to get requested resource to another URI with same method that used prior request. This has the same semantic than the 302 Found HTTP response code, with the exception that the user agent must not change the HTTP method used.
  • 308 Permanent Redirect: This means that the resource is now permanently located at another URI. This has the same semantics as the 301 Moved Permanently HTTP response code, with the exception that the user agent must not change the HTTP method used.

4xx Client Error:

  • 400 Bad Request: This response means that server could not understand the request due to invalid syntax.
  • 401 Unauthorized: Authentication is needed to get requested response. This is similar to 403, but in this case, authentication is possible.
  • 402 Payment Request: This response code is reserved for future use. Initial aim for creating this code was using it for digital payment systems however this is not used currently.
  • 403 Forbidden: Client does not have access rights to the content so server is rejecting to give proper response.
  • 404 Not Found: Server can not find requested resource.
  • 405 Method Not Allowed: The request method is known by the server but has been disabled and cannot be used. The two mandatory methods, GET and HEAD, must never be disabled and should not return this error code.
  • 406 Not Acceptable: This response is sent when the web server, after performing server-driven content negotiation, doesn’t find any content following the criteria given by the user agent.
  • 407 Proxy Authentication Required: This is similar to 401 but authentication is needed to be done by a proxy.
  • 408 Request Timeout: This response is sent on an idle connection by some servers, even without any previous request by the client. It means that the server would like to shut down this unused connection.
  • 409 Conflict: This response would be sent when a request conflict with current state of server.
  • 410 Gone: This response would be sent when requested content has been deleted from server.
  • 411 Length Required: Server rejected the request because the Content-Length header field is not defined and the server requires it.
  • 412 Precondition Failed: The client has indicated preconditions in its headers which the server does not meet.
  • 413 Payload Too Large: Request entity is larger than limits defined by server; the server might close the connection or return an Retry-After header field.
  • 414 URI Too Long: The URI requested by the client is longer than the server is willing to interpret.
  • 415 Unsupported Media Type: The media format of the requested data is not supported by the server, so the server is rejecting the request.
  • 416 Requested Range Not Satisfiable: The range specified by the Range header field in the request can’t be fulfilled; it’s possible that the range is outside the size of the target URI’s data.
  • 417 Expectation Failed: This response code means the expectation indicated by the Expect request header field can’t be met by the server.
  • 421 Misdirected Request: The request was directed at a server that is not able to produce a response. This can be sent by a server that is not configured to produce responses for the combination of scheme and authority that are included in the request URI.
  • 426 Upgrade Required: The server refuses to perform the request using the current protocol but might be willing to do so after the client upgrades to a different protocol.
  • 428 Precondition Required: The origin server requires the request to be conditional. Intended to prevent “the ‘lost update’ problem, where a client GETs a resource’s state, modifies it, and PUTs it back to the server, when meanwhile a third party has modified the state on the server, leading to a conflict.”
  • 429 Too Many Requests: The user has sent too many requests in a given amount of time.
  • 431 Request Header Fields Too Large: The server is unwilling to process the request because its header fields are too large. The request MAY be resubmitted after reducing the size of the request header fields.

5xx Server Error:

  • 500 Internal Server Error: The server has encountered a situation it doesn’t know how to handle.
  • 501 Not Implemented: The request method is not supported by the server and cannot be handled.
  • 502 Bad Gateway: This error response means that the server, while working as a gateway to get a response needed to handle the request, got an invalid response.
  • 503 Service Unavailable: The server is not ready to handle the request. Common causes are a server that is down for maintenance or that is overloaded. Note that together with this response, a user-friendly page explaining the problem should be sent.
  • 504 Gateway Timeout: This error response is given when the server is acting as a gateway and cannot get a response in time.
  • 505 HTTP Version Not Supported: The HTTP version used in the request is not supported by the server.
  • 506 Variant Also Negotiates: The server has an internal configuration error: transparent content negotiation for the request results in a circular reference.
  • 507 Insufficient Storage: The server is unable to store the representation needed to complete the request.
  • 511 Network Authentication Required: The client needs to authenticate to gain network access.

2- Response Headers

Same as Request Headers a Response Headers are also HTTP Headers that contains some metadata about the response itself. (below a table of most commonly used Response Headers)

Header Usage
Content-Type The type of the data within the response body (e.g., application/xml, application/json, text/html)
Etag Entity Tag that’s used for client-side caching (We’ll discuss what’s caching in more details later)
Last-Modified Time Stamp indicating when the last time the resource being requested (i.e., the data requested) has been modified. (Also used for caching)

3- Response Body

Same as Request Body a Response Body will contains the data the server wants to send back to the client

Figure 2- Showing Response Elements

Now i’m done for this part hope you enjoyed as i’ve said i promise to complete the series by posting a part every week so the next week i’ll be discussing REST Architecture hope you’ll enjoy it.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s