HTTP: Generating ETag Header

Question

How do I generate an ETag HTTP header for a resource file?

Answer 1

I would recommend not using them and going for last-modified headers instead.

Askapache has a useful article on this. (as they do pretty much everything it seems!)

http://www.askapache.com/htaccess/apache-speed-etags.html

Answer 2

An etag is an arbitrary string that the server sends to the client that the client will send back to the server the next time the file is requested.

The etag should be computable on the server based on the file. Sort of like a checksum, but you might not want to checksum every file sending it out.

 server                client
 
        <------------- request file foo
 
 file foo etag: "xyz"  -------->
 
        <------------- request file foo
                       etag: "xyz" (what the server just sent)
 
 (the etag is the same, so the server can send a 304)

I built up a string in the format "datestamp-file size-file inode number". So, if a file is changed on the server after it has been served out to the client, the newly regenerated etag won't match if the client re-requests it.

char *mketag(char *s, struct stat *sb)
{
    sprintf(s, "%d-%d-%d", sb->st_mtime, sb->st_size, sb->st_ino);
    return s;
}

Answer 3

How to generate the default apache etag in bash

for file in *; do printf "%x-%x-%x\t$file\n" `stat -c%i $file` `stat -c%s $file` $((`stat -c%Y $file`*1000000)) ; done

Even when i was looking for something exactly like the etag (the browser asks for a file only if it has changed on the server), it never worked and i ended using a GET trick (adding a timestamp as a get argument to the js files).

Answer 4

As long as it changes whenever the resource representation changes, how you produce it is completely up to you.

You should try to produce it in a way that additionally:

1. doesn't require you to re-compute it on each conditional GET, and
2. doesn't change if the resource content hasn't changed

Using hashes of content can cause you to fail at #1 if you don't store the computed hashes along with the files.

Using inode numbers can cause you to fail at #2 if you rearrange your filesystem or you serve content from multiple servers.

One mechanism that can work is to use something entirely content dependent such as a SHA-1 hash or a version string, computed and stored once whenever your resource content changes.

Answer 5

The code example of Mark Harrison is similar to what used in Apache 2.2. But such algorithm causes problems for load balancing when you have two servers with the same file but the file's inode is different. That's why in Apache 2.4 developers simplified ETag schema and removed the inode part. Also to make ETag shorter usually they encoded in hex:

    
<inttypes.h>
    
    
char *mketag(char *s, struct stat *sb)
{
    sprintf(s, "\"%" PRIx64 "-%" PRIx64 "\"", sb->st_mtime, sb->st_size);
    return s;
}
    

or for Java

 etag = '"' + Long.toHexString(lastModified) + '-' +
                                Long.toHexString(contentLength) + '"';

for C#

// Generate ETag from file's size and last modification time as unix timestamp in seconds from 1970
public static string MakeEtag(long lastMod, long size)
{
    string etag = '"' + lastMod.ToString("x") + '-' + size.ToString("x") + '"';
    return etag;
}

public static void Main(string[] args)
{
    long lastMod = 1578315296;
    long size = 1047;
    string etag = MakeEtag(lastMod, size);
    Console.WriteLine("ETag: " + etag);
    //=> ETag: "5e132e20-417"
}

The function returns ETag compatible with Nginx. See comparison of ETags form different servers

Answer 6

From http://developer.yahoo.com/performance/rules.html#etags:

By default, both Apache and IIS embed data in the ETag that dramatically reduces the odds of the validity test succeeding on web sites with multiple servers.

...

If you're not taking advantage of the flexible validation model that ETags provide, it's better to just remove the ETag altogether.