Would there be any interest from any of the fossil developers to work on a Knowledge Management System, which would be built on top of fossil?

The project would be open source, and would be financed.

The emphasis would be on enabling distributed teams to communicate asynchronously and collaborate on a knowledge management project.

Instead of "code", team members would collaborate on "articles".

Please let me know if there is an interest and I will gladly share more of the features and concepts.

Thank you!

Probably need to add little more context.

Some of you are probably thinking "Fossil is already a Knowledge Management System, what is this guy going on about..."

That is absolutely correct for technically minded people. The idea would be to make Fossil accessible to non-technical people.

The "code" or articles would be written in markdown or asciidoc markup that would be easy for non-technical people to edit.

Being able to move an article through "different status stages" and allow people to comment on and discuss various sections of the file would be some of the features we are looking to add.

I would be interested in seeing more on that subject, provided that it adds something more than just renaming existing concepts. There is some serious doc management happening with the Fossil project, and more so with the SQLite project.

I would suggest that the thread count on this be minimized, with good titles (such as this one already has.)

The "code" or articles would be written in markdown or asciidoc markup that would be easy for non-technical people to edit.

Are you not familiar with the fileedit option available in fossil? Same with wikiedit.

Or do you mean something akin to google docs, where you and I can be in the same file and you can see what I'm adding/editing?

Thanks Larry.

I believe Fossil already has a lot of the foundational functionality to make this work.

The best would be for me to explain the context, purpose and details, and you can then decide if any of this is worth exploring further.

(I would like to emphasize that I will be writing about high level "concepts". I am not concerned about the implementation details. My guiding principle is to try and keep things as simple a possible and leverage as much of the existing functionality as possible.)

Context

I have been looking for a good Knowledge Management System (KMS) for over 2 years now. For myself as well as my company. I also volunteer my time with several different non-profit organizations and have repeatedly noticed the lack of any good KMS to onboard new volunteers, organize and share important information, facilitate discussions, and refine knowledge over time (while preserving the discussions and context for why certain decisions were made and what alternatives were explored.)

There is nothing available that is open source, distributed, easy to use for non-technical people, and leaves people in control of their own data and knowledge.

Purpose

There would be several goals and aims, which I believe can all be addressed simultaneously if we keep them in mind from the start.

The KMS could cater for single users as well as multiple members of an organization (up to a thousand members).

For organizations the KMS would essentially become a defacto asynchronous communication platform (similar to email but better), and allow people to share ideas, participate in discussions, build and refine knowledge by (asynchronously) collaborating on articles, without having to have a "hosted" cloud solution.

A decentralized communication network, if you will.

The emphasis is on "thoughtful" communication rather than "chatting".

New members will be able to join an organization and "sync" into the main repository to get all the historical knowledge, and start participating in discussions, as well as read previous discussions and resolutions.

However the benefit and value of such a system are in the details.

Details

The value and benefit of the KMS would be derived from:

• The ability to start a discussion on any part of an article. A word, sentence, paragraph, section or the whole article itself.
• New conversations can be "spun out" of existing conversations, and their relationship and timeline will be preserved.
• The complexity comes from the fact the articles are living documents and will go through different "versions", so we need to know which version a discussion belongs to and be able to easily rebuild the chronology of a conversation across multiple versions.
• The unit of measure for a (mature) piece of knowledge is an article. The KMS would contain many articles.
• Articles can belong to different categories (similar to "tags", but much more structured).
• Organizing categories will also be an important aspect of the KMS. (Organizations would be encouraged to create their categories upfront and maintain them over time).
• Users will be able to search articles by categories as well as by normal keywords.
• Users will be able to reply to an article by writing their own article
• Users will be ale to publish their own articles on the internet using Fossil and create a real "web" of information as each article would be linked to other user's Fossil servers. (This concept might be slightly confusing. An organization could choose to publish all articles for all authors, but individual users could also choose to publish their own articles. i.e. only publish articles of which they are the author.)
• A user could belong to multiple different organizations (This might get way too complicated to manage, but will put it out there to discuss. I get the impression the "greater the challenge" the more appealing the project ;-)
• Articles could be written in Markdown or AsciiDoc. (Perhaps even LaTeX for researchers).
• An easy user interface for non-technical people would be needed (at least for Markdown)
• Articles would have a "status" category that could allow them to progress through a workflow. (From "work in progress" to "draft" to "published" etc.... Essentially the "status" is just one of many categories.)
• Being able to copy/paste images into an article would be important. (Creating UML diagrams would also be beneficial.)

I could write a lot more but I think this is enough to get a conversation going and get some initial feedback to gauge interest levels.

The emphasis should be on keeping things as simple as possible for non-technical users.

---

I would suggest that the thread count on this be minimized, with good titles (such as this one already has.)

I'm not sure I fully understand. Are you suggesting you don't want this thread to get too long? and you would prefer we create new threads to discuss sub-topics?

Are you not familiar with the fileedit option available in fossil? Same with wikiedit.

Correct I was not familiar with those. Thanks for pointing it out.

Or do you mean something akin to google docs, where you and I can be in the same file and you can see what I'm adding/editing?

I am only interested in asynchronous editing (not real time collaboration).

In this 2007 article on CMS I concluded that according to criteria not so far from your own, Mediawiki was not the answer. But Mediawiki came closest while still giving excellent chances of rescuing my data at any later date, because it has an enormous ecosystem. There are many reasons not to like Mediawiki, but it does support knowledge flow, focussed communication and an audit log sufficient for Records Management and formal information management systems. You can't force Mediawiki to have a strong permissions management system, but it seems to be good enough for a lot of organisations.

In 2022 and from recent experience, I find Mediawiki has evolved features that Fossil does not, or, Fossil does not make so accessible to use. For example, non-technical people can effectively use and write templates and use categories in an advanced way, and it has an internal mail/notification system, multiple graphing and drawing languages similar to Pikchr and the Lua scripting language.

You are right that Fossil is not so far from this either and I would be delighted if it were to grow in this direction. But I wonder if you might benefit from considering contemporary Mediawiki to be a rough prototype of what you want.

Dan

Thanks for the suggestion. You are right, MediaWiki is very close.

I believe having a proper version control system as the foundation is crucial.

The solution needs to be fully distributed for redundancy.

Each member has to have a copy of all the files locally on their machine so they can work offline, and sync their changes when they go back online.

It needs to be easy for non-tech people to use, but it also has to play nice with tech tools. I fully intend to use dendron.so as my front end to write articles.

The categories would be hierarchical categories, that could be stored in a database. I haven't seen a proper implementation of this yet, and would like to explore this further within the Fossil/SQLite context.

I have a budget. Just need to see if there are people with the skills and interest to make it a reality.

Ben, before getting to substance, I want you to know that nothing here is meant to cast any aspersion upon you or your project. My concerns are chiefly about discouraging abuse of this forum. (And I do not suggest you are there, yet.)

I believe Fossil already has a lot of the foundational functionality to make this work.

Likely so, IMO. When making it work, you will find help with Fossil available here because (ta-da!) this forum is all about Fossil and attracts participants interested in that tool.

The best would be for me to explain the context, purpose and details, and you can then decide if any of this is worth exploring further.

I'm sure much or all of it is worth further exploration. But we must ask: To whom? and: Where? Those answers may or may not generally be users of this forum, and here. Those questions are my focus here.

Purpose

...

Details

...

Those purposes and details make me ask (myself first): Is this on-topic here? It all seems to be about the feature set of a system that happens to use Fossil. The project probably uses some programming language (loosely defined), and perhaps relies on web server/browser technology. Do those facts mean this thread also belongs in fora dedicated to those technologies?

... I could write a lot more but I think this is enough to get a conversation going and get some initial feedback to gauge interest levels.

I would suggest that your tool be used to support such conversation, even if it is not yet fully featured. And posting something here, resembling "Here is an interesting use of Fossil for a doc system.", with a link to something all about that tool, would not be an abuse of this forum. I, personally, would be interested to see, and possibly use, your creation. (And, I, personally, am interested in sailing, but do not come here to discuss it.)

Certainly, issues you encounter while adapting Fossil to your system's needs are on-topic here, as are Fossil feature requests. I can assure you that many people who use a SCM are also involved in (or victims of) some doc management system(s). But that fact alone does make discussion of such a system on-topic here. You have likely noticed that, to a limited extent, Fossil is also a doc management system.¹ ("DMS") Requested enhancement to that aspect of Fossil, either to make it somewhat less limited as a DMS or better support a deluxe DMS, are perfectly on-topic here.

I would suggest that the thread count on this be minimized, with good titles

I'm not sure I fully understand. Are you suggesting you don't want this thread to get too long? and you would prefer we create new threads to discuss sub-topics?

To the extent discussion of your DMS appears here, but is only of peripheral interest to Fossil users who participate in this forum, it is better if the main form page not be littered with thread titles (linked trees of posts ostensibly on the subject in the title) that compete in volume with the threads which are clearly on-topic here. A few threads are harmless (except for storage costs), and uninterested users can easily ignore them. So, I am suggesting non-proliferation of threads.

---

1. As an old advertisement loudly said, "It slices, it dices, ... ."

In that case, I suggest seeing if those will work for you as is. The only caveat is that a file needs to exist before editing it, though. I imagine that could be as simple as having a temp.txt file, editing that and renaming it, though.

All the time we see people who are (ab)using SQLite in interesting ways, so it's possible Fossil could also be used for your use case, as is. Fossil certainly works perfectly fine for all the SQLite documentation: https://www.sqlite.org/docsrc/dir?ci=tip&name=pages

Good luck with your endeavor!

I believe having a proper version control system as the foundation is crucial.

If you have developers who are capable of working on it, mediawiki "could" have fossil's own SCM features directly integrated via the libfossil project. That library is still missing fossil's network sync protocol (and will until someone else adds it), but it has all of fossil's core SCM features and is 100% compatible with fossil. (As its primary developer and a long-time fossil developer, it's a hard requirement for me that the library is/remains compatible with fossil, though it also would be interesting to see someone fork it and take it in other directions. Perhaps yours is such a project.)

Each member has to have a copy of all the files locally on their machine so they can work offline, and sync their changes when they go back online.

Using the above hypothetical approach it would mean working with their mediawiki/libfossil combo, then running the fossil binary to perform the actual sync.

I have a budget. Just need to see if there are people with the skills and interest to make it a reality.

FWIW, the above (and future) commentary on this subject doesn't represent any statement to that affect on my part ;). Yes, i think it's an interesting and worthy project, but its scope goes well beyond anything i could commit to beyond providing tidbits of insight into fossil's internals and links to relevant code.

Regarding using Fossil as a foundation to version, publish and share mostly documentation, coupled with writing a wiki and/or combined with light and user friendly publishing format, particularly in the context of knowledge management:

• I have been using Fossil since 2011 mostly for prose instead of code, with several prototypes like the ones you can see in this Portfolio where we, at our community, use Fossil as our publishing platform for booklets, books, manuals, and wikis.

• A year ago we started to combine Fossil with TiddlyWiki as a PKM (Personal Knowledge Management) and we created a syllabus to teach a workflow with such technologies to non-coders with pretty good results. They become "power users" of such combinations, and learn how to use the terminal, Fossil, Markdown and TiddlyWiki. So the workflow is not intended for end users, but is becoming smoother as we advance and we add more automation to it via interactive executable documentation (as I think that it's mostly a missing piece of PKM, despite of its utility for end users).

• For example, there is a detailed data narrative about how we are combining Fossil, TiddlyWiki and Pharo / Glamorous Toolkit to migrate content from a static site generator (Hugo) to TiddlyWiki + Fossil. The publication uses and extended version of Markdeep that I recently adapted to export/import Lepiter interactive notebooks to/from the web.

I think that there is a lot of overlap between what we are doing over here, and what you're looking for. So, thanks for starting this conversation, as I think it helps to raise awareness where Fossil is being a key piece beyond the use of the majorities and also as I think that there is a lot of innovation in the periphery beyond the "common use case" and what majority of users have chosen. Fossil itself is and example of innovation beyond the majorities (using Git).

On a related matter, I think that an important feature of forum software is related with how it allows to focus on several topics or ignore them to cultivate signal instead of noise (Discourse and Nim forum being good examples of such feature). In both forums software, one can see a lot of indicators of healthy conversation from long threads for minorities to short thread for majorities and anything in between. Hopefully some threads like this can be nurtured publicly in places like the Fossil forum and "on-topic" doesn't become equal to "only what interest to the majorities".

Thanks Stephan. Appreciate your reply, insights and links.

I realize there is still a lot more work for me to do to clearly articulate what I have in mind.

Having these discussions helps with that process.

I believe Fossil already has all the required core functionality. The challenge is turning what is already available into a user friendly (for non-techies) communication and knowledge platform.

The key concept is being able to build, clearly represent and explore "graphs of communication".

A node can be :

• A text file
• A portion of a text file (paragraph, sentence or word)
• A wiki page
• Forum threads and posts
• Tickets

Referring to the correct text file version will be important.

The idea would be to allow a user to select any text file in the repository and display a graph of all the other nodes that are linked to it (I.e. other text files, wiki pages, or forum threads and posts). And then easily select the appropriate node to continue the conversation from.

Thanks Larry. Understood and point taken.

Will create a Fossil project to document and discuss these ideas further.

---

Your reference to a "Document system" is probably more precise than a "Knowledge Management System" (KMS).

KMS means too many different things to different people.

A more accurate description would be a "Document and Communication system".

As I mentioned elsewhere in this thread, I believe Fossil already has all the underlying core functionalities that are required to make this vision a reality. Yes, it slices and dices! really well :-)

To be clear, I have some high level concepts, but I don't have the skills nor the technical knowledge to make it a reality. That is the reason I started this post to see if people with the skill and knowledge would be interested.

I'm not even "stuck" on the high level concepts. It is more about how much value and benefits we can deliver to the end users, with as little effort as possible, re-using what we already have.

Whether it would mean adding some extra functionality to Fossil or forking the project. I have no idea. I don't even understand Fossil's architecture well enough to work out the best way forward.

I just have some ideas and a budget. I am only trying to establish some preliminary discussions to first figure out if what I have in mind is achievable and of interest to Fossil developers, and second, to develop a "win-win" situation. (i.e. Sponsor the development of some features that will benefit the Fossil community at large and deliver tangible benefits to non-technical organizations.)

Thanks Offray!

You are absolutely right, and the IndieWeb movement and tenets, have strongly influenced my thoughts.

(I can't emphasize enough how crucial the IndieWeb concepts are to this whole project.)

I have been using Fossil since 2011...

Really nice link.

A year ago we started to combine Fossil with TiddlyWiki as a PKM ...

Wow. That is really cool. Will definitely look into it more.

For example, there is a detailed data narrative about how we are combining Fossil, TiddlyWiki

Truly impressive.

I think that there is a lot of overlap between what we are doing over here, and what you're looking for.

Absolutely. Need to spend more time looking into your solution.

---

You have definitely given me a lot to read, digest and think about.

Thank you very much! :-)

Hi Ben,

Is this an open source thing you'd be working on? I too would like a CMS backed by Fossil. Well, actually many things backed by Fossil.

https://www.mail-archive.com/fossil-users@lists.fossil-scm.org/msg25067.html & https://www.mail-archive.com/fossil-users@lists.fossil-scm.org/msg25028.html being prior conversations on the broad topic.

Email me if you're not wanting to pen something public - paul@hammant.org

• Paul Hammant

[[ edit by Stephan: hyperlinked URLs ]]

Will create a Fossil project to document and discuss these ideas further.

Please do let us know where this place is when it is born. I'm very interested in this topic. I've been using various softwares for knowledge collection and management for a couple of decades. (Ward's Wiki, TWiki, DekiWiki, ..., Fossil, Joplin, Onenote) All are excellent in their own ways, and all have ultimately proved not a good fit for the purposes I've bent them. From my experience one point that should not be ignored is the friction many folks have using a plain text editing interface. Pretty matters. At very least headings, bold, italic, float aligning figures, footnote style links (instead of inline). Pasting images from clipboard helps; where they get stored and later re-used is a complicated question.

Fossil has many elements that strongly appeal to me, specifically around the gravity well of "all batteries included, both server and client, for real".

If you want "pretty" there are numerous options available already. One is the Trix editor from Basecamp. Their editor would appeal to anyone. It wouldn't even require any additions to Fossil because you could use a CDN.

Will do. Looks like there in enough "experience" from everyone that commented on this thread to come up with a workable solution.

Might take a few weeks, as I have been run off my feet, but will definitely let everyone know where to go once I get to it.

Thanks. I never heard of Trix before. Very cool. I just hope it can save content as markdown or asciidoc.

Hi Paul,

Yes it would all be open source.

I too would like a CMS backed by Fossil. Well, actually many things backed by Fossil.

I know the feeling :-)

Looked into the links you provided and concur.

---

Will email you once I get a bit more bandwidth (in the next few weeks). Cheers!

Maybe the possibility of finally making a libfossil could be considered for this project.

This kind of initiative is exactly one of those that "only needs some money to get through fast/realistically" (imho)

It might be some time (and money) wasted from your project's pov.

But I humbly think that it should be considered, maybe only because I think it would be hard to make a really stable KMS using the current APIs.

Maybe the possibility of finally making a libfossil could be considered for this project.

https://fossil.wanderinghorse.net/r/libfossil

All of the core-most SCM features, except for network sync, are implemented.

Really nice. It seems close to being made official right. !

It seems close to being made official right. !

Presumably you mean officially part of the main Fossil project, in which case the answer is unequivocally no, and there's no reason to believe that that will ever change.

libfossil is an independent project to which Richard has given his blessing and permission to steal Fossil code for, but it is in not operated under the official Fossil project umbrella.

If, on the other hand, you mean "officially that guy's pet project," then yes, it is :).

Do you really believe this project will never be made part of Fossil itself?

I would understand if you said "we are not there yet" but I can't see any reason why it ever won't.

Do you really believe this project will never be made part of Fossil itself?

As the library's architect and primary developer i have no intention of ever pushing for it to be made so, nor to back such any such pushing by others. Doing so would require putting fossil on hold for half a year or more while volunteer(s) reimplement it entirely on top of the library API. (A feature-by-feature port is not a viable option - there are too many ways for that to break badly.) i can say with the confidence borne of having spent countless hours in both fossil and libfossil that it would require a truly massive effort and provide little or no benefit to the main fossil app.

In fact, it would make fossil slower and much larger: the library's requirement that literally every single possible failure case be checked for, handled, and propagated back to the caller, instead of simply dying on error like fossil prefers to do, causes equivalent features in the library to take, on average, 2-3x as many lines of code. That's not only a maintenance burden but also introduces runtime overhead.

Fossil's lack of a library, and the freedom to output any text anywhere, call abort() at the slightest hint of a problem, not be beholden to stable APIs, and to rely on an abort-on-OOM allocator are some of the reasons features can often so easily and quickly be added to fossil. Doing those same things from a library interface requires, for example, checking every single result code/allocation, abstracting every byte of user-visible output away behind client-provided callbacks, and handling OOM conditions. Such aspects all add abstraction penalties and lines of code, as well is remove much of the freedom to change them later on because developers expect APIs to remain reasonably stable.

KIT.james (kjames3411) on 2022-03-21 12:57:00:

Do you really believe this project will never be made part of Fossil itself?

The respective principal authors of these projects don't want your proposal to happen, and so that is the first and final answer.

In addition, there are facts that show this is a good answer, including that while there are many uses for libfossil, being a building block for Fossil is not one of them. It might sound as if it should be because other projects have relationships between thing and libthing. But that is not true for Fossil.

I personally have a further view as I tried to explain earlier. The Fossil file format cannot be standardised without at least two independent implementations existing, and your question implies destroying that possibility. At the moment that remains only a potential possibility.

Imagining a future when the file format is standardised, then it would be usual for the network transport used for that format to be standardised. Fossil's network transport needs work, and perhaps this is the way to do it. The usual way of things is that once there is this sort of standardisation activity, then other implementations are developed - perhaps in other languages such as Rust, or for specialist purposes such as for very large numbers of files, or to support decentralised networking models.

Humanity currently has no standard way of storing or transferring versioned file information, despite the entire world being built on source code and other files. Many skillsets would be needed to work on more organised and more general approaches to the problem of "why is humanity relying on informal file versioning systems, or no versioning at all?"

This is a large problem; specific contributions would be welcomed.

Dan Shearer

Oh, didn't notice you were the author of libfossil.

I understood your point.

From the programmers pov though, even if the sources are never merged (never have code in common), "officializing" the project (which means, "we the Fossil staff certifies that libfossil is cared amout with the same about of love and is usable with Fossil with a high guarantee of stability") would be really nice.

In the end imho end users (programmers) do not really care about the implementation.

This is what I meant when saying "part of Fossil". I meant "part of the Fossil project".

Thank you for your answer.

I think my other post (29) answers this too.

From the programmers pov though, even if the sources are never merged (never have code in common), "officializing" the project (which means, "we the Fossil staff certifies that libfossil is cared amout with the same about of love and is usable with Fossil with a high guarantee of stability") would be really nice.

As we don't really officially have a "staff" beyond the project's founder, hoster, and Benevolent Dictator for Life, with the rest of us being volunteers who came and go at will, i'm not sure that such a statement would really do more than unduly ease some peoples' minds. It wouldn't really be conveying anything more than "marketing speak" and certainly wouldn't make the code any more stable.

As far as stability goes...

• API stability: libfossil very explicitly does not currently make any API stability guarantees for the simple reason that it's not seen enough action for me to be convinced that its current APIs are ones we'd really like to see stabilized. Lately it's been developed hand-in-hand with its biggest client app, fnc, so have some confidence that the APIs are reasonably useful, but only more widespread experience with improve that confidence level.
• Stability in the sense that "it won't break your repo" is a topic near and dear to my heart, but no amount of stating "it's stable and won't swallow your repositories" is going to make it any more stable or unlikely to swallow any repositories. It would just be more "marketing speak."

Contributions which move it closer to "stable" in any sense are of course welcomed. The more, the merrier.

Of course I mean "realize it first and when it is done state it" :)

Just stating it would be a lie.

Fossil presently is able to display differences or a timeline for a single file. If I understand what you seek, it's already there. I can attest that getting at the feature is obscure, but one way was mentioned in this forum recently, and I have done it with a series of well-placed clicks in the Web UI.

I'll pitch in my $/50 on this: It would be nice if the Web UI made the feature easier to discover. I use it from time to time, but more often get a whole diff involving many files, then ignore the output for most of them.

If I understand what you seek, it's already there

Yes, it is very close.

The key would be to filter the nodes based on a change of value for the tags contained in the files frontmatter.

It would be nice if the Web UI made the feature easier to discover

Agree.

Hi Ben,

The features you're asking to transform Fossil in to a KM can be added if Fossil is a core component of a publishing stack and is extended from outside. I'll show you an example. It doesn't include tags, but they have YAML front matter and custom links, pointing to page history.

As I told you in another thread, our use of Fossil has been mainly for documentation purposes instead of software source code, combining it with wikis and custom made solution. One if them is called Brea that is in the middle of a "decoupled CMS" and a static web site generator. Fossil acts as the storage backend and publishing part of this Brea prototype (we still need to work on friendlier/shorter URLs).

As you can see in pages like the one introducing Brea, each page has a little toolbar in the right corner, with a small clock icon, that point to the history of such page or another icon pointing to its source (Markdown) code. Almost all pages (except for the landpage) are produced programatically. Users of Brea just add Markdown/JSON files to certain locations and the (Pharo) scripts create the proper page, converting between formats, applying templates, including custom links on the pages. The templates use Mustache and are super simple to do a you can preview them on the browser to tweak them on demand.

So, my advice would be following a similar approach to the one you're sharing with us, with an important tweak: instead of extending Fossil from outside, make it part of a stack and think what features would be needed to have in Fossil, to keep the stack simple. For example, in my Brea decoupled CMS / Static site generator, powered by Fossil, I would like to have shorter urls. I imagine that I can made it by building a wrapper/router around Fossil internal URLs (maybe within my web server), but I wonder if TH1/tcl are better suited for that, inside Fossil. Such specific questions arise once I have build prototypes from the perspective of a Fossil powered stack. That perspective could be helpful for your Fossil powered KM centered around documents. Think in the experience you want to provide (i.e: documents history) and if it requires changes in Fossil or interactions inside the stack.

Cheers,

The closest thing to a knowledge management system built on top of fossil is PDQ, a CMS like Wordpress built on top of fossil using jsish. It even has plugins see PDQ Plugins.

It should be easy to combine Tiddlywiki with jsish to make a PKB too. That would be a more suitable way of using Fossil as a storage backend for TiddlyWiki.

The jsish project can be found on Github and there's also a repo with binaries.

However it seems like the original https://jsish.org website is gone, now it's been taken over by a domain squatter.
Which makes me curious as to what's happened to Peter MacDonald pmacdona, his website is offline and his Github was last updated on 12th Feb, maybe he's died of Covid-19?