Fossil Forum

Tags for wiki pages or how to organize mixed artifacts into one topic?
Login

Tags for wiki pages or how to organize mixed artifacts into one topic?

Tags for wiki pages or how to organize mixed artifacts into one topic?

(1) By smartmic on 2021-05-27 09:33:40 [source]

Is there a way to organize wiki pages, e.g. by assigning tags to them? I saw that there is a --technote-tags option in the wiki create command. Since technotes are also wiki pages, does it also apply to normal wiki pages? As far as I understood the artifact specification, there are differences though.

What I want to do is organize a project in some epics or user stories or whatever you call it and have consistent tags for all artifacts related to same categories. If I would filter the timeline for any tag, it would give me not only the check-ins related to this tag but also wiki pages, technotes, tickets etc.

Is this a reasonable approach or are there other/better ways to organize topics within a fossil repository besides the branch tags?

(2) By Stephan Beal (stephan) on 2021-05-27 15:00:51 in reply to 1 [link] [source]

Is there a way to organize wiki pages,

Not currently, no.

e.g. by assigning tags to them

Kinda. The data model supports tagging arbitrary artifacts, but we don't have UIs for tagging/filtering on anything other that checkins and the tag command is currently hard-wired for checkins.

If someone would volunteer to expand the UIs for wiki, forum posts, and tickets, the underlying infrastructure for it should already "just work." As proof of concept i just tagged a propagating tag to a wiki page in the libfossil project using its counterpart of the tag command:

https://fossil.wanderinghorse.net/r/libfossil/timeline?c=be6e66fda8

The /info page for wiki artifacts doesn't currently list tags, though, so it's not trivial to verify that the tag is actually propagating.

Is this a reasonable approach or are there other/better ways to organize topics within a fossil repository besides the branch tags?

There's not currently a really good approach. One potential partial workaround is to work with embedded docs instead of wiki pages, break them into branches, and, as you say, use branch tags or other propagating tags. Propagating tags within a single branch wouldn't help, though - all docs in the same checkin lineage share the same propagating tags.

Tagging files is also possible but our infrastructure for propagation of tags does not (insofar as i recall) know how to handle that type of propagation (file inheritance works much differently from checkin/wiki/forum/ticket inheritance).

(3) By Stephan Beal (stephan) on 2021-05-27 16:54:15 in reply to 2 [link] [source]

If someone would volunteer to expand the UIs for wiki, forum posts, and tickets, the underlying infrastructure for it should already "just work." As proof of concept i just tagged a propagating tag to a wiki page in the libfossil project using its counterpart of the tag command:

Reminders to self:

The /ci_tags page understands arbitrary artifacts and shows the proof-of-concept tag added to that one version of that wiki page, but does not show it as having propagated to later edits. The implication is that propagation doesn't really work on non-checkins. This seems to be caused by tag_propagate() expecting that the parentage is stored in plink, but that's currently only the case for checkins. Whether or not wiki heritage can be stored in plink without a rat's tail of unwanted side effects is not yet clear (but it "should" be okay to do so). In manifest_crosslink(), tag_propagate_all() is currently only run for checkins which have a parent. That call would need to be extended to include wiki/forum/tickets which have a parent, too.

(4) By anonymous on 2021-05-29 21:53:50 in reply to 2 [link] [source]

Non-propagating tags should be able to tag any artifact (whether or not it is a deck, although it might not be as useful for non-decks).

Propagating tags should propagate to any artifact with the P card with the hash of the one that it is propagating from, so that later versions of the same thing will have the same tags. With the standard formats of Fossil, this means the manifest, wiki, technote, and forum. (It does not apply to tickets, which have their own key/value pairs, separate from tags. If you apply a tag to a ticket, it is non-propagating for a specific ticket change (unless a future or non-standard implementation allows tickets to have P cards (perhaps in order to allow comments to be edited)).)

In my opinion, applying propagating tags to non-decks should not be allowed, since the above won't work for non-decks. Tags for non-decks will be non-propagating if they are used at all (although I don't know if the system will necessarily understand the tags; it might or might not).

(5) By Andy Bradford (andybradford) on 2021-05-29 22:22:54 in reply to 4 [link] [source]

> Non-propagating tags  should be able  to tag any artifact  (whether or
> not it is a deck, although it might not be as useful for non-decks).

What's a "deck"?

Thanks,

Andy

(7) By Stephan Beal (stephan) on 2021-05-29 22:30:51 in reply to 5 [link] [source]

What's a "deck"?

A "collection of cards."

i believe this anonymous poster is the same one who's working on their own implementation of a db schema for use with fossil's data model. We've interacted in the past and it came to light that we had both independently settled on the term "deck" for what the fossil-internal API calls Manifest.

(6.1) By Stephan Beal (stephan) on 2021-05-29 22:32:52 edited from 6.0 in reply to 4 [link] [source]

Non-propagating tags should be able to tag any artifact (whether or not it is a deck, although it might not be as useful for non-decks).

They "should," yes, but the tag command is currently hard-coded to handle only checkins. Patches improving that would be gladly considered. For testing of this change i'm currently using libfossil's tag command, which doesn't care what type of artifact it's working on.

Propagating tags should propagate to any artifact with the P card with the hash of the one that it is propagating from, so that later versions of the same thing will have the same tags.

"Should," yes, and i just checked in a patch for that in the past hour, but it needs significant testing to ensure that checkin-related algos are not unduly affected. Historically, the only records in the plink table have been checkins, but now wiki, forum, and technote records (as you say, anything with a P-card) are put into there. There's a chance, however, that some older algos may expect only checkins to be there and backfire.

(It does not apply to tickets, which have their own key/value pairs, separate from tags. If you apply a tag to a ticket, it is non-propagating for a specific ticket change (unless a future or non-standard implementation allows tickets to have P cards (perhaps in order to allow comments to be edited)).)

Funnily enough, the patch initially tried the same for tickets because i thought that tickets had a P-card. That they don't seems weird to me, but their data model is different than the other record types: a ticket is composed of a base record and then any number of "patches" applied to it. They get applied (unless i'm sorely misremembering) in time order, rather than following a chain of P-cards.

In my opinion, applying propagating tags to non-decks should not be allowed, since the above won't work for non-decks.

We cannot support propagating tags of plain files without infrastructure specific to those, so that seems unlikely to happen. All deck types which have a P-card, though, can be handled uniformly using the existing schema and algorithms. All that is needed is to add those entries to the plink table (which the above-linked patch now does).

(Edit: fixed link)

(8) By Stephan Beal (stephan) on 2021-05-29 22:56:08 in reply to 6.1 [link] [source]

"Should," yes, and i just checked in a patch for that in the past hour, ...

For completeness's sake, here's a demonstration of a propagating wiki tag which was cancelled somewhere along the line to test that propagation properly ends:

# SQL to list all versions of a given wiki page and their tags:
$ cat WikiTag.sql 
SELECT
datetime(event.mtime, 'localtime') AS timestamp,
substr(uuid,1,12) AS uuid,
(SELECT group_concat(tagname, ', ') FROM tag, tagxref
        WHERE tag.tagid=tagxref.tagid 
        AND tagxref.rid=blob.rid AND tagxref.tagtype>0) as tags
FROM event, blob
WHERE
blob.rid in (
         select srcid from tagxref
         where tagid=(select tagid from tag where tagname='wiki-FIXMEs')
)
AND blob.rid=event.objid
ORDER BY event.mtime DESC
;

# Output of that query...
$ f-query -e WikiTag.sql 
timestamp	uuid	tags
...<snip>...
2021-03-22 18:55:33	fccf58660787	wiki-FIXMEs
2021-03-22 15:39:32	43aecd7b71d8	wiki-FIXMEs
2021-03-20 07:36:47	0010305624f0	wiki-tag-propagation-test, wiki-FIXMEs
2021-03-20 05:17:35	8e2ba08496fe	wiki-tag-propagation-test, wiki-FIXMEs
2021-03-11 16:35:17	4988ca8ac764	wiki-tag-propagation-test, wiki-FIXMEs
2021-03-11 15:59:46	63379677c635	wiki-tag-propagation-test, wiki-FIXMEs
2021-03-04 20:15:12	22289337881a	wiki-tag-propagation-test, wiki-FIXMEs
2021-03-02 09:30:27	0f48096d4b06	wiki-tag-propagation-test, wiki-FIXMEs
2021-03-01 11:24:45	7e152f416905	wiki-tag-propagation-test, wiki-FIXMEs
2021-03-01 04:03:17	7116cf36eda1	wiki-tag-propagation-test, wiki-FIXMEs
2021-02-27 06:23:08	320a6c2b92a5	wiki-tag-propagation-test, wiki-FIXMEs
2021-02-27 05:03:02	e3ed1c816607	wiki-tag-propagation-test, wiki-FIXMEs
2021-02-26 18:37:51	ce297002218b	wiki-tag-propagation-test, wiki-FIXMEs
2021-02-26 05:58:48	31dd3124c998	wiki-FIXMEs
2021-02-26 04:38:15	a6d0b4d09655	wiki-FIXMEs
...<snip>...

(9) By Stephan Beal (stephan) on 2021-06-04 05:11:27 in reply to 1 [link] [source]

Is there a way to organize wiki pages, e.g. by assigning tags to them?

Follow-up: we have since added the missing infrastructure bits which were necessary for propagating tags for non-checkins, so fossil can now propagate tags for all "structural artifact" types which have direct child/parent relationships: checkins, wiki pages, forum posts, and technotes. This required no core data format changes, just some additional db records in an already-existing table.

What we cannot propagate (and likely never will) are tags to plain opaque client-side files. Fossil can tag anything which has a hash, but propagation requires tracking the parent/child lineage, and the tracking of that for plain files is significantly different than the above-listed artifact types. It "would be possible," but there's currently no pressing interest in doing so. (That said, it "would be cool" to be able to add propagating tags to embedded docs, which are opaque client-side files.)

As part of this change, the tag command's add/cancel subcommands now support tagging these artifact types. The find/list subcommands still need to be reevaluated and updated for this support. (That's on my TODO list for the 2.16 release.)

What we're missing is UI elements for viewing, manipulating, and filtering tags on the non-checkin artifact types. Adding arbitrary tags to, e.g., wiki pages and forum posts, has to be done with care because fossil's tags are case-sensitive: foo-bar and Foo-Bar are two completely unrelated tags and humans are inconsistent in how they tend to enter such data. Thus any UIs related to adding such tags need to be capable of showing users any existing tags and allowing selection of those, as well as entry of free-form tags and detection/warning of case-only collisions (possibly automatically/silently adjusting to existing case when a collision is detected).

There is currently no estimate on when/whether such UI elements will be integrated. Anyone who feels interested in working on them is more than welcome to do so! (My personal recommendation would be to add a new JSON-based API, similar to how /chat is implemented, as that it can be easily reused in wiki, forum, and technote pages.)

Sidebar: tickets can also have tags but the way ticket edits are managed is completely different from wiki/forum/technote, so propagation of tags is unnecessary for them. Whereas each version of a wiki/forum post/technote is a completely separate artifact containing the artifact's complete text content along with a link back to its parent version, a ticket is stored as a single base artifact and each change is made as a "patch" (of sorts) to that. There is no parent/child relationship in ticket history, and thus no tag propagation.

(10) By smartmic on 2021-06-04 20:40:50 in reply to 9 [link] [source]

Hi Stephan,

These are really great news, thank you very much for this implementation!

I think this fits very well into the concept of tight integration of all SCM features which fossil offers. Personally, I am glad you picked up the idea and I am looking forward to use (and test) it for my repos.

—Martin