Infinite loop in DELTA table
By andygoth on 2018-09-22 00:18:13 [link]
I'm having problems cloning the SQLite repository. I get an infinite loop error, as well as virtually empty repository (despite being 20 megabytes).
$ f version This is fossil version 2.2 [613fe1b1fb] 2017-04-19 07:08:01 UTC $ f clone http://server/sqlite/ sqlite.fossil Round-trips: 3 Artifacts sent: 0 received: 13104 Clone done, sent: 912 received: 19401205 ip: 22.214.171.124 Rebuilding repository meta-data... 100.0% complete... Extra delta compression... Vacuuming the database... project-id: 2ab58778c2967968b94284e989e43dc11791f548 server-id: 02b60258c7ddda07b2551f01f0673f63fa51b74c admin-user: user (password is "password") $ f sync -R sqlite.fossil Sync with http://server/sqlite/ Round-trips: 1 Artifacts sent: 0 received: 208 Fossil internal error: infinite loop in DELTA table $ f info -R sqlite.fossil project-name: SQLite project-code: 2ab58778c2967968b94284e989e43dc11791f548 check-ins: 1 $ f info -R sqlite.fossil trunk uuid: 704b122e5308587b60b47a5c2fff40c593d4bf8f 2000-05-29 14:16:00 UTC tags: trunk comment: initial empty check-in (user: drh) $ f rebuild -stats sqlite.fossil 100.0% complete... Artifacts: 13062 Manifests: 1 Clusters: 0 Tags: 0 Wikis: 0 Tickets: 0 Attachments: 0 Events: 0 Other: 13061 $ f sync -R sqlite.fossil Sync with http://server/sqlite/ Round-trips: 1 Artifacts sent: 0 received: 208 Fossil internal error: infinite loop in DELTA table
I'm trying to switch to a newer Fossil version, but I'm having trouble with that.
The actual most recent check-in for my SQLite repository I just cloned is
d10e63629183f6daf0c263cd4dae052a3786c8c1480b3b6a73124b3315e41951. It displays
just fine on the server, which is running Fossil version
By drh on 2018-09-22 15:45:53
Can you get me a copy of the fossil repo that is showing the delta loop, so that I can investigate the problem?
Have you tried running "fossil test-integrity" on the repo that shows the problem?
By andygoth on 2018-09-26 01:53:34 [link]
Here is the repository. Please let me know when you've downloaded it because I don't intend to leave it up forever.
test-integrity has heartburn like you couldn't believe. Wrong hash on every artifact, it seems, plus a number of skipped phantoms. At the end, it says:
13062 non-phantom blobs (out of 13316 total) checked: 12001 errors low-level database integrity-check: ok
I am using Fossil client version 2.2 [613fe1b1fb] to clone SQLite from Fossil server version 2.7 [5fdd53120e]. The SQLite repository's latest trunk version is 42e04fefbc241dd33f12abd66344a87720ae4cda 2018-09-25 13:51:31 UTC.
By andygoth on 2018-09-26 03:49:58 [link]
Amazingly, the problem also occurs when using 2.7 [5fdd53120e] on both the client and the server! This is most disturbing.
I wonder if there's something corrupt in the sqlite.fossil on the server. Yet, I'm able to log directly into the server and open it without issue. When run on the server, test-integrity takes over five minutes, having to chew through 79182 artifacts rather than just the relative handful that made it (corrupted?) into the clone. Here's the output:
skip phantom 73940 f04ded1d9b40d54463162264e37e6d92411d09427eea592ef05681035e2f2e64 skip phantom 73941 9dd591ef24b302a5fe2af0619d0cda6733348bacc541b3c0a134ac25981d4b2a skip phantom 73942 5981969cad5afebb8f8098489d0419cc9ead560a0f22a484230f1886011cd57c skip phantom 75591 469694857a5301e1da8cf067017d400403f3b12882d4265cefb3fd185c536305 skip phantom 75592 db32506880e727597e8629c21d29a4b5e3360bec7548d98ecdbb09685fd6e7a8 skip phantom 78638 caee77d9a8e1da2551cef44ba7a11e2de36eb8c047273148b4126bfc1edd21da skip phantom 79127 d103c041ccb3a009926b6aa34a283a7cb8e4a645711ecd7a3002a90558d02e9d skip phantom 79128 a3e9e62c6cafac5b5f103edcff77085b463085fca60c515aa16723c3d0006d44 skip phantom 79129 dcac190277fab9c6a9bdf651d18b25d97e0db49b3cc709ca145403c8b556bce3 skip phantom 79130 8e59dc7807d15d4fca6ee7b6a624447748eb24afb48f902fca27c2d550d84f1c skip phantom 79131 4889866915b8b8fc28ca20cfc3f2ed4890eb1f0712b0537298fbdd498ce41510 skip phantom 79132 43c3d2d707a7c54dcb75660be9347d6bd79c4844f81f6ed73446570876f5f72b skip phantom 79133 4b575b8050cdb703df26d28228a60b88f60e5279453ba71c65e0739575609a0b 79169 non-phantom blobs (out of 79182 total) checked: 0 errors low-level database integrity-check: ok
Huh, by the way, I just noticed MSIE11 automatically corrects the capitalization of "sqlite" to "SQLite". It would appear SQLite has become important to Microsoft.
By andygoth on 2018-09-26 04:44:14 [link]
If I scp the repository to the target machine and then clone it from one file to another, it's fine.
If I run a server process on the target machine hosting the scp'ed copy then attempt to clone from that via http://localhost:8080/, it fails.
If I rebuild the repository on the server then clone again to the target machine via HTTP, it still fails.
If I log into the server and clone the repository file to another, it's fine, just like when I do this on the target machine.
If I log into the server and clone the repository HTTP URL to a file, it fails.
In all cases, having the HTTP server in the middle seems to be the cause of failure. The clone operation gets 13100 artifacts then stops.
Also curious: the repository file is 198 megabytes, a successful clone is 84 megabytes, and an unsuccessful clone is 20 megabytes.
By jungleboogie on 2018-09-26 05:00:07 [link]
I can't clone: $ fossil clone https://andy.junkdrome.org/misc/sqlite.fossil andy.fossil server says: 404 Not Found Clone done, sent: 279 received: 509 ip: 126.96.36.199 server returned an error - clone aborted I can download, but the file stops at the ~20MB mark: $ ftp https://andy.junkdrome.org/misc/sqlite.fossil Trying 188.8.131.52... Requesting https://andy.junkdrome.org/misc/sqlite.fossil 100% |******************************************************************************************************************************************************| 20096 KB 00:49 20578304 bytes received in 49.35 seconds (407.20 KB/s)
By andygoth on 2018-09-26 05:21:11 [link]
I'm not running a Fossil server on andy.junkdrome.org, just hosting the repository file per drh's request.
As for stopping at the 20 megabyte mark, that's the whole problem. That's as much as I get when I try to clone from the computer that's actually running the Fossil server within my internal network, which I can't expose.
By drh on 2018-09-26 12:57:21 [link]
What webserver are you using? Is it somehow configured to limit the size of HTTP replies? Are there rewriting rules? Have you tried cloning using https: instead of http: to help ensure that nobody in the middle is corrupting the data coming over the wire?
By andygoth on 2018-09-26 16:20:56 [link]
I'm using Fossil's own web server, that's all. No other web server is running.
I have the following in my /etc/inetd.conf:
80 stream tcp nowait.1000 root /home/fossil/fossil/fossil /home/fossil/fossil/fossil http -host SERVERNAME -repolist /home/fossil
Also, /home/fossil/fossil/fossil is hilarious. :^) The username is fossil, the binary is in the fossil subdirectory, and the binary is called fossil.