Fossil Forum

Hash values used by style_load_one_js_file()
Login

Hash values used by style_load_one_js_file()

Hash values used by style_load_one_js_file()

(1) By Stephan Beal (stephan) on 2020-05-10 09:10:28 [link] [source]

i just now learned about style_load_one_js_file().

https://fossil-scm.org/fossil/info/6d1c5f7e3bec91f4?ln=703-705

which a function implemented for /fileedit almost duplicates.

style_load_one_js_file() uses the compile-time fossil manifest version as its cache-buster, which means that the cache buster never changes during a given development session, even if that builtin file's contents change. The fileedit-related impl instead uses an md5 hash prefix of the builtin file being served.

Would there be any objection to my changing it to use a hash of the builtin, rather than the fossil version? That would make it functionally equivalent to:

https://fossil-scm.org/fossil/info?udc=1&ln=1590-1602&name=d7490c0d31e01bc0

(2) By Florian Balmer (florian.balmer) on 2020-05-10 13:31:27 in reply to 1 [link] [source]

Calculating the hash will impose a runtime penalty -- now you have to weigh if this is more serious than pressing Ctrl+F5 during development sessions ... ;-)

(5) By Richard Hipp (drh) on 2020-05-10 14:13:15 in reply to 2 [link] [source]

A reasonable point. My new fossil_exe_id() also involves running an MD5 hash, even if it is a small one.

Perhaps I should modify the mkversion.c utility as follows:

  • Adjust the makefiles so that mkversion is rerun for every recompile, even if the "manifest" is unchanged.

  • Add a new #define in the output from mkversion: FOSSIL_EXE_ID. This new #define would have a hash based on MANIFEST_UUID and the current timestamp.

Then if I convert fossil_exe_id() to use the FOSSIL_EXE_ID #define instead of rerunning the MD5 hash each time, we effectively move that hash from run-time to compile-time.

If mkversion runs on every build, that means that a small amount of recompilation has to occur every time you do "make". But on my workstation, that extra recompilation only takes about 500 milliseconds, so that doesn't seem like too big of a problem.

(6) By Stephan Beal (stephan) on 2020-05-10 14:22:09 in reply to 5 [link] [source]

The once-per-hit cost of a 50-100 byte md5 doesn't sound all that bad to me. My initial thought was to hash the builtins when they're compiled and add that to their struct, but the builtin-maker binary is standalone (no access to fossil's hashing routines) so we'd need to use a custom hash (which isn't a big deal - they don't have to be collision-free or even necessarily more than 32 bits). If that sounds like a halfway sane thing to do, i don't mind implementing it.

(7) By Stephan Beal (stephan) on 2020-05-10 14:25:32 in reply to 6 [link] [source]

In fact, we don't even really need to hash the builtins - it would probably suffice to use their file timestamp as the hash.

(8) By Stephan Beal (stephan) on 2020-05-10 14:36:46 in reply to 6 [link] [source]

How does this sound: i modify the builtins generator to add each file's mtime to the struct, and we just use that mtime as-is for the cache buster? The one tiny catch there is that i can't test stat() on Windows, so would need to copy/paste a few lines from file.c and hope that it works on Windows.

(9) By Richard Hipp (drh) on 2020-05-10 14:58:58 in reply to 8 [link] [source]

I think just use fossil_exe_id() to get your cache= value. The value returned by fossil_exe_id() is now computed at compile-time.

Doing so means that if you change a file, you'll need to touch manifest before you rebuild in order to cause a new hash to be generated. But that does not seem to be too great of a problem.

(10) By Stephan Beal (stephan) on 2020-05-10 15:01:05 in reply to 8 [source]

How does this sound: i modify the builtins generator to add each file's mtime to the struct, and we just use that mtime as-is for the cache buster? The one tiny catch there is that i can't test stat() on Windows, so would need to copy/paste a few lines from file.c and hope that it works on Windows.

Nevermind: that would also require, for Windows, access to fossil_utf8_to_path(), which brings along its own baggage :/. Oh, well.

We "could" use the time() that mkbuiltin is run instead. That would "expire" all cache-busters every time a single builtin changes, though, but that's not a big deal for local development and doesn't impact deployed builds.

We also have the option of custom-hashing each builtin as its processed.

(3) By Richard Hipp (drh) on 2020-05-10 14:00:47 in reply to 1 [link] [source]

See check-in 54a8243bf5e4b24b. Maybe use the new fossil_exe_id() function to get a unique identifier string. You will have to touch the etag.c file when you rebuild with a change but don't commit, but that doesn't seem so bad, since you only need to do so when you are making some changes that affects caching.

(4) By Stephan Beal (stephan) on 2020-05-10 14:10:03 in reply to 3 [link] [source]

See check-in 54a8243bf5e4b24b.

Perfect, with no per-request penalty :).