HTTP Cross-Origin Requests in JavaScript

Since I started working with front-end development, a large part of the projects I’ve handled have been based on a separation of front-end and back-end, where the back-end only provides APIs, and the front-end renders the actual pages based on these APIs. I personally think this is a pretty good model, with clear division of responsibilities, allowing both front-end and back-end to focus on their respective modules, and it also gives the front-end more room to maneuver.

Unlike the previous template-based approach, after separating the front-end and back-end, most communication between them is done by the front-end actively sending requests to the back-end. Most of these requests are made using Ajax, which is a very convenient way to fetch data. However, when Ajax encounters cross-origin requests, things can become quite complicated. This article mainly outlines some of the cross-origin request issues I have encountered during project development, and it will also include some background knowledge about cross-origin requests. PS: There’s a little Easter egg at the end of the article! 😊

Strictly speaking, a cross-origin request is not just limited to Ajax requests; it refers to any request made by a page to resources from a different domain. For example, an tag with a source from another domain, or third-party CSS stylesheets included in the page.

For images and CSS, cross-origin requests do not pose significant security issues since these requests are read-only and do not have side effects on the source resources. However, if the cross-origin request is initiated from a script, due to the high flexibility of scripts, browsers impose restrictions based on the same-origin policy for security reasons, meaning that scripts can normally only request resources from the same origin. If a page needs to request resources from another site via a script, it should operate under the Cross-Origin Resource Sharing (CORS) mechanism.

Wait a minute, what is the same-origin policy?

Same-Origin Policy

For two pages (or resources), they are considered to comply with the same-origin policy if they meet the following three conditions:

1. The protocol is the same

2. The port is the same

3. The domain is the same

Additionally, about:blank and javascript: inherit the origin of the page that loads them. Resources from data: have an empty security context.

Moreover, subdomains can set document.domain via JavaScript to comply with the same-origin policy. For example:

On a page from the subdomain http://a.example.com/test.html, if document.domain is set to ‘example.com’ via JavaScript, then the current page will comply with the same-origin policy with http://example.com/page.html.

Simply put, for the page http://www.example.com/page1.html, the following pages do not comply with the same-origin policy, and scripts cannot directly request these resources:

• https://www.example.com/page1.html: Different protocol

• http://www.example.com:81/page1.html: Different port

• http://another.example.com/page1.html: Different domain

So, what is CORS?

CORS (Cross-Origin Resource Sharing)

CORS essentially specifies a set of HTTP headers to determine whether a script can perform a cross-origin request. Before understanding these request headers, let’s first look at the types of cross-origin requests.

Requests made via scripts can be done in two ways: one is through creating an XMLHttpRequest, and the other is through the fetch API.

Generally speaking, cross-origin requests can be roughly divided into two types, one of which is called simple requests, which meet the following conditions:

• The request method is one of GET, POST, or HEAD.

• Besides the request headers automatically added by the browser (like Connection, User-Agent, etc.), only the following request headers are allowed:

• Accept

• Accept-Language

• Content-Language

• Content-Type

• The value of the Content-Type request header can only be one of application/x-www-form-urlencoded, multipart/form-data, or text/plain.

If any of the above three rules are violated, then it is not a simple cross-origin request. Non-simple cross-origin requests differ from simple cross-origin requests in that before sending the request, the browser will first send a preflighted request to confirm whether the subsequent request is allowed by the server.

Preflight Requests

In actual project development, when using XHR or the fetch API to request interfaces, many times additional special request headers or special HTTP methods like PUT or DELETE (common in RESTful APIs) are included. Due to the additional request headers or the use of special HTTP methods, the browser considers these requests as non-simple cross-origin requests and will automatically send a preflight request (an OPTIONS request) before the actual request is sent.

The OPTIONS request will send the special HTTP request headers and HTTP request methods used in the current cross-origin request to the server, such as Access-Control-Request-Method and Access-Control-Request-Headers. After receiving the OPTIONS request, the server responds with the corresponding response headers. The browser then uses the returned response headers to determine whether the cross-origin request is allowed. The actual request will only be sent if the browser determines that the OPTIONS request has passed. Below is an example of an OPTIONS request and the actual GET request’s response headers and request headers:

OPTIONS /api4 HTTP/1.1Host: us1.serenader.me:3333Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
Access-Control-Request-Method: PUT
Origin: http://us1.serenader.me:3334User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.95 Safari/537.36Accept: */*
Referer: http://us1.serenader.me:3334/
Accept-Encoding: gzip, deflate, sdch
Accept-Language: zh-CN,zh;q=0.8,en;q=0.6,zh-TW;q=0.4,fr;q=0.2

HTTP/1.1 200 OK
X-Powered-By: Express
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET,POST,PUT,DELETE
Content-Type: text/html; charset=utf-8Content-Length: 2ETag: W/"2-REvLOj/Pg4kpbElGfyfh1g"Date: Thu, 19 Jan 2017 15:21:15 GMT
Connection: keep-alive

PUT /api4 HTTP/1.1Host: us1.serenader.me:3333Connection: keep-alive
Content-Length: 0Pragma: no-cache
Cache-Control: no-cache
Origin: http://us1.serenader.me:3334User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.95 Safari/537.36Accept: */*
Referer: http://us1.serenader.me:3334/
Accept-Encoding: gzip, deflate, sdch
Accept-Language: zh-CN,zh;q=0.8,en;q=0.6,zh-TW;q=0.4,fr;q=0.2

HTTP/1.1 200 OK
X-Powered-By: Express
Access-Control-Allow-Origin: *
Content-Type: text/html; charset=utf-8Content-Length: 2ETag: W/"2-REvLOj/Pg4kpbElGfyfh1g"Date: Thu, 19 Jan 2017 15:21:15 GMT
Connection: keep-alive

After understanding simple cross-origin requests and non-simple cross-origin requests that send preflight requests, let’s take a look at which HTTP headers determine the ‘fate’ of these cross-origin requests.

To help readers better understand the role of these HTTP headers, I have created a simple demo, which is open-sourced on GitHub. If you’re interested, you can check the code at this link or visit this online demo to preview the effect: http://us1.serenader.me:3334/. Remember to open the Chrome console after loading the page to see detailed request information.

Access-Control-Allow-Origin

Access-Control-Allow-Origin is a response header that specifies which domain scripts are allowed to request the current resource.

Cross-origin requests (whether simple or non-simple) will include the Origin request header to indicate which domain is making the request. At this point, the server’s response headers must include an Access-Control-Allow-Origin header that matches the Origin request header; only then can the cross-origin request possibly succeed. Otherwise, it will fail.

Access-Control-Allow-Origin is the first hurdle. The matching rules for its value are:

• If its value is the wildcard *, then all domains are allowed to make cross-origin requests.

• If its value is a specific fixed domain, then only that domain is allowed to make cross-origin requests, and other domains will fail.

• If its value is a domain with a wildcard, such as *.example.com, then that domain and its subdomains are allowed to cross-origin.

Specific demos can show the situation when a script requests an interface without configured cross-origin headers, where the request is intercepted by the browser:

Demo-1 shows the situation when the interface has configured the Access-Control-Allow-Origin response header, but it does not match the domain of the script request; in this case, the browser will report this error:

Only requests with correctly configured Access-Control-Allow-Origin response headers can receive responses normally, as shown in demo-2, where the request headers and response headers are:

GET /api2 HTTP/1.1Host: us1.serenader.me:3333Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
Origin: http://us1.serenader.me:3334User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.95 Safari/537.36Accept: */*
Referer: http://us1.serenader.me:3334/
Accept-Encoding: gzip, deflate, sdch
Accept-Language: zh-CN,zh;q=0.8,en;q=0.6,zh-TW;q=0.4,fr;q=0.2

HTTP/1.1 200 OK
X-Powered-By: Express
Access-Control-Allow-Origin: *
Content-Type: text/html; charset=utf-8Content-Length: 2ETag: W/"2-REvLOj/Pg4kpbElGfyfh1g"Date: Thu, 19 Jan 2017 15:03:33 GMT
Connection: keep-alive

For simple cross-origin requests, usually just the Access-Control-Allow-Origin response header is needed for the request to succeed (not considering cases with cookies, which will be discussed later). When the request is not a simple cross-origin request, the situation becomes more complicated.

Access-Control-Allow-Headers

Access-Control-Allow-Headers is used to inform the browser which special request headers are allowed for the current interface. This HTTP header typically appears in the response headers of OPTIONS requests.

When a request sets a special request header and the requested interface does not configure the Access-Control-Allow-Headers response header, the following error will be reported, as shown in demo-3:

The screenshot above shows that the request included a custom X-Custom-Header request header, but the request failed during the preflight phase. To successfully complete the request, the Access-Control-Allow-Headers: X-Custom-Header must be configured in the OPTIONS request’s response.

Access-Control-Allow-Methods

Similar to the previous HTTP header, Access-Control-Allow-Methods informs the browser which HTTP methods are allowed to request the current interface. This HTTP header is also meaningful only in the response headers of OPTIONS requests. When this response header is absent, an error like this will be reported:

Again, the screenshot shows that the request failed during the preflight phase. To successfully execute the request, the Access-Control-Allow-Methods: GET,POST,PUT response header needs to be configured.

Access-Control-Max-Age

Due to the presence of OPTIONS requests, for a non-simple request, the actual request will involve two requests. This can waste bandwidth since this validation should only occur the first time. Once validated, subsequent requests to the same interface do not need to trigger another OPTIONS request.

Fortunately, there is a response header called Access-Control-Max-Age that can achieve this functionality. This header specifies how long a successful preflight request will be cached by the browser, reducing the actual requests and bandwidth waste.

Access-Control-Allow-Credentials

By default, any cross-origin request will not include any credentials, which include:

• Cookies

• Authentication-related requests

• TLS client certificates

However, in most cases, we need to include cookies in requests, so the withCredentials option for cross-origin requests needs to be enabled.

To manually enable the transmission of cookies, the following methods can be used:

• XHR: Set xhr.withCredentials = true for the XHR object.

• Fetch: Enable credentials in the passed options like this: fetch(url, { credentials: ‘include’ }).

Once withCredentials is enabled, the request will automatically include cookies when sent.

However, besides needing to manually enable withCredentials on the front-end, the server also needs to have the corresponding response header support for the request to be successful.

The Access-Control-Allow-Credentials response header indicates whether the current requested resource allows credentials to be included. The request will only succeed if its value is true; otherwise, it will fail, as shown below:

You can refer to demo-7 to see the request and response headers.

Additionally, once the withCredentials option is enabled, the server’s Access-Control-Allow-Origin response header cannot be a wildcard; it must be a specific domain, or else the request will fail. The specific error message is as follows:

Demo-8 and demo-9 demonstrate the cases when requests include cookies with the response header configured as a wildcard and correctly configured for a specific domain, respectively.

Conclusion

In summary, when making requests in scripts, the following situations can arise:

1. If the protocol, port, or domain of the requested resource matches the address of the page making the request, then it complies with the same-origin policy, and the request can be sent normally. Conversely, it is considered a cross-origin request and must comply with the CORS mechanism.

2. For all cross-origin requests, the server must return the Access-Control-Allow-Origin response header, and its value must match the Origin request header in the request. Only then can the request be allowed; otherwise, it will be intercepted by the browser.

3. Cross-origin requests are divided into two types: simple cross-origin requests and non-simple cross-origin requests. For non-simple cross-origin requests, the browser will first send a preflight request (an OPTIONS request) to verify whether the server allows access to the requested resources. If the OPTIONS request succeeds, the actual request will be sent; otherwise, the request will fail at the OPTIONS stage, and the subsequent actual request will not be sent.

4. When a request includes special request headers, the server’s response to the OPTIONS request must include the Access-Control-Allow-Headers response header, and its value must include the names of the special request headers included in the request. Only then can the request succeed; otherwise, it will be intercepted by the browser.

5. When a request uses special HTTP methods, the server’s response to the OPTIONS request must include the Access-Control-Allow-Methods response header, and its value must include the current HTTP method being used. If this response header is absent, or if the current method is not included in its value, the request will be intercepted by the browser.

6. Because non-simple requests actually involve two requests each time, to reduce the number of OPTIONS requests and bandwidth waste, the server can configure Access-Control-Max-Age to specify how long the browser can cache the OPTIONS request, allowing subsequent requests to the same interface without sending another OPTIONS request.

7. When cross-origin requests need to include cookies or other credentials, the withCredentials option must be manually enabled, and the server must configure the Access-Control-Allow-Credentials response header; otherwise, the request will not include any credentials, or if Access-Control-Allow-Credentials is not present, the request will be intercepted by the browser.

8. When requests include credentials, the server must configure the Access-Control-Allow-Credentials response header, and the value of the Access-Control-Allow-Origin response header cannot be a wildcard; it must be a specific domain. Otherwise, the request will be intercepted by the browser.

Among these eight points, it is worth noting points three and eight.

The OPTIONS request is a key point that is easily overlooked; some back-end developers often only know to write the Access-Control-Allow-Origin in the response headers of the interface without realizing the existence of OPTIONS requests. Especially since OPTIONS requests are not sent with every cross-origin request, it leads to some confusion about why a GET request is sent while an OPTIONS request is also sent. Even if cross-origin permissions for OPTIONS requests are configured, it is still easy for requests to fail due to the absence of the corresponding Access-Control-Allow-Headers or Access-Control-Allow-Methods response headers.

Point eight is also a very important key point. If you have interfaces that need to provide services to multiple different domain websites, then your interface cannot use cookies or other credentials, since Access-Control-Allow-Origin cannot be set to a wildcard, limiting the objects that can use the interface.

Easter Egg Time

Earlier, it was mentioned that only non-simple requests would trigger OPTIONS requests, and simple requests meet only those three conditions. However, the reality is not as perfect as imagined.

If you use XMLHttpRequest to implement file uploads and add any event listeners to the xhr.upload object, it will trigger an OPTIONS request, even if the request itself meets the three conditions for a simple request. Once the event listeners are removed, it does not trigger.

This ‘bug’ was something I inadvertently discovered while writing the uploader library. I initially thought it was a browser bug, but later found out on StackOverflow that it was actually a hidden ‘feature’ of the browser.

Turns out this is not a bug. The spec for XMLHttpRequest does mention that upload progress event handlers should cause the ‘force preflight’ flag to be set. I was a bit confused when this was not specifically mentioned in the CORS spec, even though that spec does reference the existence of a ‘force preflight’ flag.

Source: https://segmentfault.com/a/1190000008456994