I am trying to use fossil to manage a bunch of large binary files. Reading through the forum, this is clearly an unsupported usage scenario, however the only documentation I can find about a file size limit is around 2GB. Some forum posts cite the sqlite maximum blob size of just over 2GB. Some mentioned that fossil itself has a defined hard limit of 2GB, and will abort if given a file larger than that. Regardless, the consensus as far as I have found is that files under 2GB should work fine.

I am getting an error when trying to commit a file somewhere in between 1.15 and 1.27GB (as reported by macOS). The 1.27GB file will give the following error:

./Skyrim - Textures4.bsa is more than 20,000,000 bytes in size.
Commit anyhow (a=all/y/N)? y
New_Version: ede60a8821bd3086bf3cb85ef6cfb8862ea8ea2711b85b54702a86cd1858dabb
ERROR: [drive_c/Program Files (x86)/Steam/steamapps/common/Skyrim Special Edition/Data/Skyrim - Textures4.bsa] is 1265325033 bytes on disk but 0 in the repository
NOTICE: Repository version of [drive_c/Program Files (x86)/Steam/steamapps/common/Skyrim Special Edition/Data/Skyrim - Textures4.bsa] stored in [file-8394e8cb55324f1f]
working check-out does not match what would have ended up in the repository:  6de330b84d88162b151d71df3a2cd159 versus 75589727aec84eba0905a83f85103225

Testing with a 3gb test file gives an error closer to what I would expect:

out of memory
Transaction started at ./src/checkin.c:2478 never commits

Is there some other limit to file size around 1.2GB that I am hitting?

Also thank you so much for this amazing software! I have used git for years and still don't really understand it. I have now been playing with fossil for less than a week and already feel like I understand how to use it.

Try recompiling with -DSQLITE_MAX_LENGTH=2000000000. The default value is 1000000000. The upper bound is 2147483647, or something close to that.

I can't seem to get this to work, but I don't think I am correctly adding it during the compilation. What is the proper way to add the flag? I tried make DSQLITE_MAX_LENGTH=2000000000 and adding it to CFLAGS and TCCFLAGS in the Makefile. Do I need to compile sqlite from source with the option and provide that binary to fossil? Just in case it makes a difference, I am using xcode c compiler on an arm mac.

That did the trick! Thanks for helping me out with that!

What is the proper way to add the flag?

Not sure this is "proper" but this works:

$ ./configure ...
$ make CFLAGS="-DSQLITE_MAX_LENGTH=2000000000"

That should arguably be CPPFLAGS=... but the build is not set up to inherit CPPFLAGS from the CLI (but will accept CFLAGS from the CLI).