Lesson 18 beginner

HTTP, HTTPS and the Web

The application layer and the client-server model, how the World Wide Web works, HTTP requests and responses, methods, status codes and headers, HTTP/1.1 vs HTTP/2 vs HTTP/3, cookies, caching, HTTPS, and how content delivery networks make sites fast, all with curl.

The application layer is where networking meets the programs people use. Every protocol here solves one user-facing job: fetch a page, send an email, copy a file, look up a name. The most important of them is HTTP, the protocol of the web and of almost every API you will ever call.

The client-server model

Most application protocols follow the same pattern:

  • A server listens on a well-known port and waits.
  • A client connects, sends a request and reads the response.
  • The server handles many clients at once and keeps the shared data.

The model is simple to secure and scale: you add servers behind a load balancer, cache responses near users, and keep clients thin. Its weak spot is that the server is a central dependency, which is why serious services run many of them in several places.

The World Wide Web

The web, invented by Tim Berners-Lee at CERN in 1989, rests on three ideas:

  1. URLs to name every resource.
  2. HTML to describe documents that link to other documents.
  3. HTTP to fetch them.

A URL packs a lot of networking into one line:

https://blog.jefersonjeronimoai.online:443/tutorials/network/?ref=nav#faq
scheme   host (resolved by DNS)        port  path              query    fragment

The fragment (#faq) never leaves the browser; it only scrolls the page.

An HTTP exchange

HTTP/1.1 is plain text, which makes it easy to read. curl -v shows both sides:

$ curl -v https://blog.jefersonjeronimoai.online/tutorials/ -o /dev/null
* Connected to blog.jefersonjeronimoai.online (…) port 443
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
> GET /tutorials/ HTTP/1.1
> Host: blog.jefersonjeronimoai.online
> User-Agent: curl/8.11.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: nginx
< Content-Type: text/html; charset=UTF-8
< Vary: Accept-Encoding
< strict-transport-security: max-age=31536000; preload; includeSubDomains
< X-Content-Type-Options: nosniff

Lines with > are the request; < is the response. The request line has a method, a path and a version. The Host header lets one IP serve many sites. The status line says how it went, and headers describe the body that follows.

Methods

MethodPurposeSafeIdempotent
GETRead a resourceYesYes
HEADLike GET but headers onlyYesYes
POSTCreate or trigger processingNoNo
PUTReplace a resourceNoYes
PATCHChange part of a resourceNoNo
DELETERemove a resourceNoYes
OPTIONSAsk what is allowed (CORS preflight)YesYes

Idempotent means repeating the request has the same effect as doing it once, so clients and proxies may safely retry it after a network error.

Status codes

ClassMeaningCommon codes
1xxInformational101 Switching Protocols (WebSocket), 103 Early Hints
2xxSuccess200 OK, 201 Created, 204 No Content
3xxRedirect301 Moved Permanently, 302 Found, 304 Not Modified, 308 Permanent Redirect
4xxClient error400 Bad Request, 401 Unauthorized, 403 Forbidden, 404 Not Found, 429 Too Many Requests
5xxServer error500 Internal Server Error, 502 Bad Gateway, 503 Service Unavailable, 504 Gateway Timeout

For network people, 502 and 504 are the interesting ones: a proxy or load balancer could not get a good answer from the server behind it (502: bad or refused, 504: too slow). The problem is between the proxy and the backend, not with the client.

$ curl -s -o /dev/null -w '%{http_code} %{time_namelookup} %{time_connect} %{time_appconnect} %{time_starttransfer} %{time_total}\n' https://blog.jefersonjeronimoai.online/
200 0.012 0.043 0.069 0.801 0.812

That one line splits the time into DNS (12 ms), TCP connect (43 ms), TLS done (69 ms), first byte (801 ms) and total (812 ms). The network part is done in 69 ms; the remaining 730 ms before the first byte is the server building the page. If the gap between TLS and first byte is large, the server is slow; if connect is large, the network is.

HTTP versions

VersionTransportKey change
HTTP/1.0TCPOne request per connection
HTTP/1.1TCPKeep-alive connections, Host header, chunked transfer
HTTP/2TCP + TLSBinary framing, many streams multiplexed on one connection, header compression
HTTP/3QUIC over UDPIndependent streams (no TCP head-of-line blocking), faster setup

The meaning of methods, status codes and headers is the same in all of them; only the wire format changes.

Statelessness and cookies

HTTP is stateless: every request stands alone. To remember a login, the server sends a cookie with Set-Cookie, and the browser returns it with each following request. Good cookie hygiene uses Secure (HTTPS only), HttpOnly (not readable by JavaScript) and SameSite (limits cross-site sending).

Caching

The fastest request is the one that never reaches the server. Response headers control who may keep a copy and for how long:

  • Cache-Control: max-age=86400 — reuse for a day.
  • Cache-Control: no-store — never cache (private data).
  • ETag / Last-Modified — the client can ask “has it changed?” with If-None-Match and get a tiny 304 Not Modified instead of the full body.
$ curl -sI https://blog.jefersonjeronimoai.online/wp-includes/js/jquery/jquery.min.js | grep -iE 'cache-control|etag|last-modified'
Last-Modified: Mon, 28 Aug 2023 17:14:23 GMT
ETag: "64ecd5ef-15601"
Cache-Control: public, max-age=31536000, no-transform

A static file like this can be cached for a full year (max-age=31536000), and the ETag lets the browser revalidate it cheaply.

HTTPS

HTTPS is HTTP inside TLS on port 443 (lesson 16). Browsers mark plain HTTP as “Not secure”, and features such as HTTP/2 in browsers, service workers and geolocation require HTTPS. Two headers finish the job:

  • A 301 redirect from http:// to https://.
  • HSTS (Strict-Transport-Security: max-age=31536000), which tells the browser to never try plain HTTP for that site again.

Content delivery networks

Light in fiber covers about 200 km per millisecond, so a user in São Paulo talking to a server in Frankfurt (about 10,000 km away) pays roughly 100 ms per round trip before any server work. A CDN puts copies of content in hundreds of locations near users.

How it works:

  1. Your DNS points the site to the CDN (often a CNAME).
  2. The CDN’s DNS or anycast routing (the same IP announced from every location) sends each user to the nearest edge server.
  3. The edge serves cached content directly. On a miss, it fetches from your origin server once and caches it for the next users.

Benefits: lower latency, less load on the origin, absorbing traffic spikes and DDoS attacks, and TLS terminated close to the user. Response headers usually reveal a CDN and whether you hit the cache:

$ curl -sI https://cdn.example.com/app.js | grep -iE '^(server|x-cache|cf-cache-status|age):'
server: cloudflare
cf-cache-status: HIT
age: 1874

HIT (or x-cache: Hit from cloudfront on other CDNs) means the edge answered from cache; MISS means it had to go to the origin. age says how many seconds that copy has been cached.

Exercise

A load balancer returns 504 Gateway Timeout. Where should you look first?

  1. At the backend servers, which did not answer the load balancer in time
  2. At the user’s browser cache
  3. At the DNS TTL of the site
  4. At the client’s TLS version
Show answer

Answer: A. 504 is produced by a proxy or gateway that waited too long for the upstream server. Check backend health, slow queries and the timeout settings between the two.

Exercise

Why can a client safely retry a PUT request after a network error but not always a POST?

  1. PUT requests are encrypted and POST requests are not
  2. POST requests are always cached
  3. PUT is idempotent, so repeating it has the same effect, while a repeated POST may create a duplicate
  4. PUT uses UDP and POST uses TCP
Show answer

Answer: C. Idempotent methods (GET, PUT, DELETE) can be repeated without changing the result. Repeating a POST might, for example, place an order twice.

Exercise

How does a CDN usually send each user to a nearby edge server?

  1. By asking the user to pick a region
  2. With DNS answers or anycast routing that lead to the closest location
  3. By redirecting every request through the origin first
  4. By using a larger MTU
Show answer

Answer: B. CDNs use geo-aware DNS or announce the same anycast IP from many locations, so the network itself delivers users to the nearest edge.

Key takeaways

  • The client-server model: clients send requests, servers listen and respond.
  • HTTP messages are a request line or status line, headers and an optional body.
  • Methods have meanings (safe, idempotent); status classes 2xx to 5xx tell you who failed.
  • HTTP/2 multiplexes over TCP; HTTP/3 does it over QUIC; semantics stay the same.
  • Caching headers and CDNs bring content close to users; curl -v and curl -w show where time goes.

FAQ

What is the difference between 401 and 403?

401 Unauthorized means “who are you?”: authentication is missing or invalid. 403 Forbidden means “I know who you are, and you may not do this.”

Why do I get a 301 when I request the site?

The server is redirecting you, usually from http to https or from www to the bare domain. Use curl -L to follow redirects and curl -I to see the Location header.

Is a REST API different from the web?

It uses the same HTTP: methods, status codes, headers. The responses are data (usually JSON) for programs instead of HTML for people.

Do I need a CDN for a small site?

Not necessarily. A single well-configured server with page caching is fast for nearby visitors. A CDN pays off when your audience is spread across continents or you expect traffic spikes.

↑ ↓ navigate Enter open Esc close