Fossil hook - howto
(1.1) Originally by anonymous with edits by Warren Young (wyoung) on 2020-08-30 01:27:40 from 1.0 [link] [source]
Hi all,
I want to create some hooks, to deploy something after a commit on the server side. But I'm unsure about to configure sth. like that.
I first looked at the fossil hook help page.
I then found this page with the search.
Are these two documents describing the same feature?
Can I create a hook with fossil hook add --command COMMAND --type TYPE
where command is "echo test" and type is "after-receive"?
Which type is appropriate to deploy a website with a static site generator after a commit for example? In git I would use the server side post-receive hook.
Thanks for your help.
(WY edit: URL fixes.)
(2) By Richard Hipp (drh) on 2020-08-29 19:37:58 in reply to 1.0 [link] [source]
Hooks are currently experimental. Please sign up to the Forum so that you can get email notifications, then hang out here and help us to make the hooks awesome. :-)
(12) By anonymous on 2020-09-24 08:49:17 in reply to 2 [link] [source]
Thanks for the long awaited feature and here's an end user perspective.
I am using fossil-2.12.1 and the content displayed by fossil help hook is similar to https://fossil-scm.org/fossil/help?cmd=hook , so I assume the current state is this.
IMHO it misses a detail that is very important for a newcomer. It mentions the --type TYPE argument, but doesn't provide any insight on what are the valid TYPE values. The only example
fossil hook edit --type disabled 2
doesn't shed much additional light on this.
As someone who doesn't follow fossil development on a daily basis, but who has been missing git commit hooks for years, I still search about it every couple of months. The SEO ecosystem is almost entirely full of git, so I was not able to find info on fossil hooks using a search engine. Fortunately, I also searched directly in the fossil forum, so here I am.
This topic mentions the very useful https://fossil-scm.org/fossil/doc/trunk/www/hooks.md , which answers my questions, but also makes more obvious the missing piece in the built-in help.
As an end user I would prefer to see the hook types briefly documented in the output of fossil help hook. And a reference to the aforementioned https://fossil-scm.org/fossil/doc/trunk/www/hooks.md for more details.
Thank you.
(3) By Warren Young (wyoung) on 2020-08-30 01:41:48 in reply to 1.1 [link] [source]
Are these two documents describing the same feature?
Yes.
where command is "echo test"
Given that the command is executed with a wrapper around system(3), that should work, though of course it will do nothing useful.
The main problem I anticipate is that you'll then be wanting to write a single-line shell script, which almost certainly won't work because Fossil's wrapper around system(3) refuses to run the command if it has anything funny-looking in it.
You avoid that by writing a program instead and calling that. You can write it in any language that has an interpreter on your system.
This can be a two-line script (shebang and one-liner) or a 10000 line C program, or whatever. The main restriction on language is that the hook mechanism be able to do the I/O needed to get the info Fossil is trying to pass it. In the case of after-receive, it primarily means the script has to be able to read artifact IDs from its standard input.
Which type is appropriate to deploy a website with a static site generator after a commit
You'd use after-receive if the receiving Fossil server is the one you want to run the static site generator and deploy the site.
If you want that build and deployment process to run on the sending machine — i.e. where the edit was done — then I think you'd want to set the before-commmit hook on the local machine's clone of the repo instead. This will give you the opportunity to abort the commit if the build and deployment fail, which is probably a good thing. Why allow the commit if it doesn't build and deploy successfully?
(4) By MBL (RoboManni) on 2020-08-30 08:52:00 in reply to 3 [link] [source]
My experience with the hook system is based on windows and fossil version 2.13 [88ff2642d3] 2020-08-25 00:01:25 UTC, which as of today is the most recent binary for download.
before-commit
The hook script from before-commit gets fired and runs after command execution but before the commitment itself is done; and it will be done only if the return value of the script does not indicate an error (0 = success, other values are error numbers). - The availability of the commit comments was under discussion already - in my current version they are not yet available to the hook script but with the latest checkin from drh they should be in. (Sorry, I am strill struggling with my tool chain to built myself an executable from the checked out source; so I cannot describe most recent behaviour yet.)
Once the hook script succeeded the commitment will be aborted or done with the possible autosync to the default remote-url connected repository.
after-receive
After reception of a new version the after-receive hook script becomes "fired". BUT do not expect immediate execution like it has happened with the before-commit hook script - this will not happen straight and immediately. Maybe the missing point and subject for confusions. (My expectation at the beginning was also to see the script running immediately.)
I recommend to also introduce a new hook type after-commit for local side, which gets fired and runs immediate before the autosync to remote repository happens like the remote after-receive would do; with the difference to stop autosync from happening in case of error code return from the script. (Read also my erlier post)
The hook script execution will get stored and cached only at first, waiting for executing with a delay. The hook scripts execution will be only executed by a command like fossil backoffice. Or a running cron-job of the OS which does this call on a regular time base or a running daemon, which can be executed by a command like fossil backoffice --poll 60. There is also some configuration which takes care by default that an execution of waiting hook script will have a minimum delay of one minute.
There is not yet a windows service support existing for this backoffice tasks like exists with the fossil winsrv for the repository web server. I asked for this in an earlier post already.
(5.1) By MBL (RoboManni) on 2020-08-30 10:08:53 edited from 5.0 in reply to 3 [source]
The hook system or fossil-scm 2.13-preview on Windows OS
starting command files content
fossil hook list -R D:/REPO/Project.fossil
start /min fossil server D:/REPO/Project.fossil
start /min fossil backoffice --poll 15 -R D:/REPO/Project.fossil
the hook list
Without having used the parameter substitution for %R
D:\fosdir>fossil hook list
0: type = before-commit
command = D:/UTL/lua53.exe D:/REPO/fosbuild.lua before-commit %F %A 0
sequence = 10
1: type = after-receive
command = D:/UTL/lua53.exe D:/REPO/fosbuild.lua after-receive F=%F A=%A 0 --
sequence = 20
2: type = after-receive
command = D:/UTL/fossil.exe update --setmtime --debug -v
sequence = 50
D:\fosdir>
my hook script fosbuild.lua
I wrote a small Lua script to show what's all the inputs are. I used the lua53.exe executable from the ZeroBrane editor for this.
print("Time", os.date() )
print( "Program", arg[-1] ) -- first parameter (not counted)
print( "Script", arg[0] ) -- 2nd parameter (not counted, too)
local rc = 0
for n,item in ipairs( arg ) do -- print all counted parameters
print( n, item )
end
if arg[#arg]=="--" then -- check if last parameter indicates 'stdin used'
for line in io.lines() do -- read all lines from stdin
print( line )
end
rc = tonumber(arg[#arg-1]) or 0 -- use last parameter as return value
else
rc = tonumber(arg[#arg]) or 0 -- use last parameter as return value (no stdin usage)
end
local function listFile(id,fname)
if id == "before-commit" then
for line in io.lines(fname) do
print( line )
end
end
end
listFile( arg[1], arg[3] )
print( "return", rc )
return rc
This script runs for any hook script, means for now before-commit as well as after-receive. The difference is the last command line parameter --, which expects and reads input from stdin or does not.
example logging output from hook type=before-commit script
This hook script expects input from a file.
before-commit called from inside of lua53.exe fosbuild.lua 0
n= 1, item= before-commit
n= 2, item= D:\UTL\fossil.exe
n= 3, item= D:/fosdir/commit-description-a2ca6b64d8794f2c.txt
n= 4, item= 0
D:/fosdir/commit-description-a2ca6b64d8794f2c.txt= checkout D:/fosdir/
D:/fosdir/commit-description-a2ca6b64d8794f2c.txt= repository D:/REPO/Project
D:/fosdir/commit-description-a2ca6b64d8794f2c.txt= user MBL
D:/fosdir/commit-description-a2ca6b64d8794f2c.txt= branch trunk
D:/fosdir/commit-description-a2ca6b64d8794f2c.txt= tags trunk
D:/fosdir/commit-description-a2ca6b64d8794f2c.txt= EDITED README.md
return 0
example logging output from hook tpye=after-receive script
This hook script expects input from stdin.
after-receive called from inside of lua53.exe fosbuild.lua 0
n= 1, item= after-receive
n= 2, item= F=D:\UTL\fossil.exe
n= 3, item= A=
n= 4, item= 0
n= 5, item= --
return 0
Do not wonder that not more got logged as script output but I force-committed empty without changes but only comments; it was just to see the hook script get fired and executed. The comments are also not easily accessible from within the after-receive hook script. Keep also in mind that commitments from many users at once may have to be handled (as we are in a distributed system) in that script (other than an after-commit script would have to do); see my previous post.
I guess this is enough to start playing
To be continued...
(6.2) By MBL (RoboManni) on 2020-09-16 06:45:02 edited from 6.1 in reply to 5.1 [link] [source]
Edit: moved the description -especially of unexpected behaviour- to the top of the post
Using fossil version: 2.13 [5c90832d79] 2020-09-06 14:49:24 UTC, where checkin-comment is supposed to be part of the file handed over in the hook call before-commit.
Good news:
The availability of the comments before-commit is as intended by checkin [4ff4e1c8] to fossil-scm trunk
BAD news:
The return value of the hook script now is ignored.
I tested with a return value of 1, which should indicate an error and the abort of the fossil commit step was expected, but it happened.
The commit should have worked only with a return value of 0 - isn't it? (And as I remember it was working before like this) Can somebody please check the correct use of the return value !? (0 to continue, others to abort the pending commitment)
In these following lines I am also skeptical because I merged into the branch PLANT and NOT trunk. The trunk was ONLY used for the initial empty checkin.
[0] D:/Plant/Server/commit-description-32393cf918e2fdf8.txt= branch trunk
[0] D:/Plant/Server/commit-description-32393cf918e2fdf8.txt= tags PLANT
Another open point on my wish list
- is the introduction of a new hook type after-commit, which
- should be fired only AFTER the local commit has succeeded but BEFORE the fossil sync becomes called.
A return value of non-zero should stop the fossil sync from being performed; automated more commits may be intended to happen before the final synchronization into the public happens for them all at once. (My commitments for each DEVICE should remain non-synced until after the cherrypick merges was successful.)
It should be possible in before-commit to not only receive the comments but also to be able to add automated additional text into the commitment. I guess the checkin comments should remain somewhere in a temporary file and its file path also given to the hook script. The commitment itself then should re-read the temporary comments text file for the final commitment (maybe it is like then, in which case just the filename of the temporary file is remaining unknown).
Here the main steps in their details:
D:/Plant/Server>fossil hook list
0: type = before-commit
command = D:/UTL/lua53.exe D:/REPO/fosbuild.lua before-commit %F %A -R=%R 1
sequence = 10
...
D:\Plant/Server>fossil open D:/REPO/Device-files.fossil L7 --force --keep --nested
project-name: Device-files
repository: D:/REPO/Device-files.fossil
local-root: D:/Plant/-Server/
config-db: C:/Users/MBL/AppData/Local/_fossil
project-code: 3c03bf7ad4cc6f57c062bae6355ab111b28c5f0b
checkout: 3184521f5aab571ca6236cdb89a92ac1f3460e37 2020-09-10 18:30:41 UTC
parent: 3b5c709e1a85263ac2aef9d6b7d4454ed22a474b 2020-09-10 06:05:30 UTC
tags: L7
comment: from Device L7 (user: MBL)
check-ins: 229
D:\Plant/Server>fossil commit -m "from Device L7"
Time 09/11/20 12:38:06
Program D:/UTL/lua53.exe
Script D:/REPO/fosbuild.lua
lsqlite 3.32.3x.534.100
[1] before-commit called from inside of lua53.exe fosbuild.lua 0
[0] n= 1, item= before-commit
[0] n= 2, item= D:/UTL/fossil.exe
[0] n= 3, item= D:/Plant/Server/commit-description-5f6fbc5faf874f3a.txt
[0] n= 4, item= -R=D:/REPO/Device-files.fossil
[0] n= 5, item= 1
[0] D:/Plant/Server/commit-description-5f6fbc5faf874f3a.txt= checkout D:/Plant/Server/
[0] D:/Plant/Server/commit-description-5f6fbc5faf874f3a.txt= repository D:/REPO/Device-files.fossil
[0] D:/Plant/Server/commit-description-5f6fbc5faf874f3a.txt= user MBL
[0] D:/Plant/Server/commit-description-5f6fbc5faf874f3a.txt= branch trunk
[0] D:/Plant/Server/commit-description-5f6fbc5faf874f3a.txt= tags L7
[0] D:/Plant/Server/commit-description-5f6fbc5faf874f3a.txt= EDITED L7_Finding.txt
[0] D:/Plant/Server/commit-description-5f6fbc5faf874f3a.txt= EDITED L7_Results.txt
[0] D:/Plant/Server/commit-description-5f6fbc5faf874f3a.txt= EDITED L7_Info.txt
[0] D:/Plant/Server/commit-description-5f6fbc5faf874f3a.txt= checkin-comment
[0] D:/Plant/Server/commit-description-5f6fbc5faf874f3a.txt= from Device L7
[1] return 1
New_Version: 33a8da2b6a13997e572f75131aecd47f7a2d9d26
D:/Plant/Server>fossil close
D:/Plant/Server>fossil open D:/REPO/Device-files.fossil PLANT --force --keep --nested
project-name: Device-files
repository: D:/REPO/Device-files.fossil
local-root: D:/Plant/Server/
config-db: C:/Users/MBL/AppData/Local/_fossil
project-code: 3c03bf7ad4cc6f57c062bae6355ab111b28c5f0b
checkout: 25525b8b963346b92d305d9c613ce201ad62c7ad 2020-09-10 18:30:48 UTC
parent: 07cfce484a4673cc02c5873fb272702bef30a83d 2020-09-10 06:05:37 UTC
tags: PLANT
comment: cherrypick uploads from DEVICES into branch PLANT (user: MBL)
check-ins: 230
D:/Plant/Server>fossil merge --cherrypick L1
MERGE L1_Finding.txt
MERGE L1_Results.txt
MERGE L1_Info.txt
"fossil undo" is available to undo changes to the working checkout.
...
D:/Plant/Server>fossil merge --cherrypick L7
MERGE L7_Finding.txt
MERGE L7_Results.txt
MERGE L7_Info.txt
"fossil undo" is available to undo changes to the working checkout.
D:/Plant/Server>fossil commit -m "cherrypick uploads from DEVICE into branch PLANT"
Time 09/11/20 12:38:11
Program D:/UTL/lua53.exe
Script D:/REPO/fosbuild.lua
lsqlite 3.32.3x.534.100
[1] before-commit called from inside of lua53.exe fosbuild.lua 1
[0] n= 1, item= before-commit
[0] n= 2, item= D:/UTL/fossil.exe
[0] n= 3, item= D:/Plant/Server/commit-description-32393cf918e2fdf8.txt
[0] n= 4, item= -R=D:/REPO/Device-files.fossil
[0] n= 5, item= 1
[0] D:/Plant/Server/commit-description-32393cf918e2fdf8.txt= checkout D:/Plant/Server/
[0] D:/Plant/Server/commit-description-32393cf918e2fdf8.txt= repository D:/REPO/Device-files.fossil
[0] D:/Plant/Server/commit-description-32393cf918e2fdf8.txt= user MBL
[0] D:/Plant/Server/commit-description-32393cf918e2fdf8.txt= branch trunk
[0] D:/Plant/Server/commit-description-32393cf918e2fdf8.txt= tags PLANT
[0] D:/Plant/Server/commit-description-32393cf918e2fdf8.txt= EDITED L1_Finding.txt
[0] D:/Plant/Server/commit-description-32393cf918e2fdf8.txt= EDITED L1_Results.txt
[0] D:/Plant/Server/commit-description-32393cf918e2fdf8.txt= EDITED L1_Info.txt
...
[0] D:/Plant/Server/commit-description-32393cf918e2fdf8.txt= EDITED L7_Finding.txt
[0] D:/Plant/Server/commit-description-32393cf918e2fdf8.txt= EDITED L7_Results.txt
[0] D:/Plant/Server/commit-description-32393cf918e2fdf8.txt= EDITED L7_Info.txt
[0] D:/Plant/Server/commit-description-32393cf918e2fdf8.txt= CHERRYPICK 097a95877b405c8494873daf9c914cfb84aea522
...
[0] D:/Plant/Server/commit-description-32393cf918e2fdf8.txt= CHERRYPICK 33a8da2b6a13997e572f75131aecd47f7a2d9d26
[0] D:/Plant/Server/commit-description-32393cf918e2fdf8.txt= checkin-comment
[0] D:/Plant/Server/commit-description-32393cf918e2fdf8.txt= cherrypick uploads from DEVICES into branch PLANT
[1] return 1
New_Version: a2ebc1a62e9b41a95e839115a9919ac34f5a0d0d
D:/PlantServer>fossil status
repository: D:/REPO/Device-files.fossil
local-root: D:/Plant/Server/
config-db: C:/Users/MBL/AppData/Local/_fossil
checkout: a2ebc1a62e9b41a95e839115a9919ac34f5a0d0d 2020-09-11 10:38:13 UTC
parent: 25525b8b963346b92d305d9c613ce201ad62c7ad 2020-09-10 18:30:48 UTC
tags: PLANT
comment: cherrypick uploads from DEVICES into branch PLANT (user: MBL)
(7.1) By MBL (RoboManni) on 2020-09-13 11:21:28 edited from 7.0 in reply to 6.1 [link] [source]
My wish list for the hook system (and for my project)
1) The introduction of a new hook type after-commit, which should be fired only AFTER the local commit has succeeded but BEFORE the fossil sync becomes called.
A return value of non-zero should stop the fossil sync from being performed; automated more commits may be intended to happen before the final synchronization into the public happens for them all at once.
The hook script should get to know the newly created check-in hash id, e.g. by the manifest.uuid per parameter and the content of the manifest file per stdin (similar to how the remote hook after-receive would get called after sync).
2) It should be possible in hook script before-commit to not only receive the comments but also to be able to add automated additional text into the commitment comments. I suggest the checkin comments should remain somewhere in a temporary file (if not already handed in per file) and its file path also given to the hook script. The commitment itself then should re-read the temporary comments text file for the final commitment.
3) fossil grep search capability per web interface. The search should be possibly limited by a time range and/or to given branch name(s)
See also 3. on To Do List; just add also web-enabled usability to it.
I tried with fossil commit -M comments-file.txt but when the before-commit script was appending text it was unfortunately not yet included in the commitment.
I need this additional step between commitment steps and before going live into the wild with the final data preparation (to generate and add meta data required to support a future search method per fossil bisect for a reporting per CGI scripts over a long running timeline).
See also more about my project
(8) By MBL (RoboManni) on 2020-09-21 16:16:07 in reply to 6.2 [link] [source]
BAD news:
The return value of the hook script now is ignored.
I tested with a return value of 1, which should indicate an error and the abort of the fossil commit step was expected, but it happened.
The commit should have worked only with a return value of 0 - isn't it? (And as I remember it was working before like this) Can somebody please check the correct use of the return value !? (0 to continue, others to abort the pending commitment)
Can someone please check before next public built why the hook return code is ignored?
When non-zero the commitment should be stopped and not completed; that's how it is also described:
If any before-commit hook returns a non-zero exit code, then the commit is abandoned. All before-commit hooks must exit(0) in order for the commit to proceed
This does not happen anymore 8-(
(9) By Warren Young (wyoung) on 2020-09-21 17:05:37 in reply to 8 [link] [source]
the hook return code is ignored
I don't use hooks, but the code seems to be written as you wish it to work. By exiting with a fatal error before the checkin's DB transaction is committed, the commit will roll back.
The key line of code in that section was last touched on 2020-08-29. Are you running an older version?
(10) By MBL (RoboManni) on 2020-09-21 17:52:38 in reply to 9 [link] [source]
fatal error yes, but return code non-zero is ignored, the commit continues anyhow regardless of which return value comes back from the hook script. However, I see what you mean by showing me that code portion; it should work... maybe my script does not do what I expect it to do... I will double check. The hint about version might be valid as well ... I am using a search path but on the other hand I also have several copies in subfolders as well, so it could be that an older version was unexpectedly taken. As I said, I will double check and make sure to use the latest public version 2.13.
Thanks for this hint and the feedback
(11) By Warren Young (wyoung) on 2020-09-21 19:24:03 in reply to 10 [link] [source]
To be clear, I mean that Fossil exits with a fatal error on getting a nonzero return from your script. I am not talking about “fatal errors” in your script.