Fossil Forum

About to merge the forum-v2 branch
Login

About to merge the forum-v2 branch

About to merge the forum-v2 branch

(1) By Richard Hipp (drh) on 2018-07-31 13:44:33 [link] [source]

I am about ready to merge the forum-v2 branch into trunk. If there are any objections, voice them quickly.

The forum-v2 branch implements a "forum" capability. Here is a quick summary:

  • Messages are organized into threads. The initial post of a thread contains a title. All subsequent posts omit the title, but have an in-reply-to field.
  • Passers-by can post anonymously, or (depending on the configuration) they can create a new account on-the-spot and post using their new account. The "self-register" feature has been in Fossil for ages out of mind. I forget who contributed it. (It did not come from me.) Self-registration has been seldom used before now, as far as I know.
  • Messages can be edited, either by the user who originally posted the message, or by an administrator.
  • Each message is an artifact, using a new artifact class called "Forum". See https://www.fossil-scm.org/fossil/doc/forum-v2/www/fileformat.wiki#forum for a description. There are three new cards: G, H, and I.
  • Because each message is an artifact, forum content syncs just like everything else.
  • Forum message from untrusted users (ex: anonymous) can be held for moderation. Such messages are not synced nor may they be replied too, until they are approved by a moderator.
  • Email notification is available for new forum posts. Currently, the alert emails contain a very brief synopsis of the post - essentially just the title. This can be enhanced later to provide the complete text of the post, if that is seen as desirable.
  • Each forum message has a mimetype. Currently supported mimetypes are text/plain, text/markdown, and text/x-fossil-wiki. New mimetypes can be added later

The intent is to replace this mailing list, as well as various other mailing lists (fossil-users, sqlite-users, sqlite-dev, sqlite-announce) with the new forum feature. I hope to shut down the mailing lists and bring the forums all live within about a week. So if you have concerns, voice them soon.

(2) By Dan Barbarito (danbarbarito) on 2018-07-31 13:59:01 in reply to 1 [link] [source]

Great work, DRH! This seems like such a useful feature. Yet another reason Fossil is better than all the alternatives out there.

(3) By Richard Hipp (drh) on 2018-07-31 14:03:44 in reply to 2 [link] [source]

Thanks for testing out the auto-registration and reply mechanism :-)

(4) By Dan Barbarito (danbarbarito) on 2018-07-31 14:06:49 in reply to 3 [link] [source]

Any time. Thanks for working so hard on pushing this feature out.

(5) By Richard Hipp (drh) on 2018-07-31 14:16:58 in reply to 1 [link] [source]

There appear to be some long delays - up to 60 seconds - when submitting a post. I do not yet know what is causing this. I am working the problem. In the meantime, please continue to exercise the system and be patient if submitting a reply, or starting a new thread, takes a little longer than seems reasonable.

(8) By Dan Barbarito (danbarbarito) on 2018-07-31 14:27:51 in reply to 5 [link] [source]

I also experienced a long delay after clicking "Preview" and not "Submit". I'm assuming these issues are related but I figured I'd let you know.

(10) By Richard Hipp (drh) on 2018-07-31 14:34:03 in reply to 8 [link] [source]

I have temporary hacked around the problem, so things should be working better for the moment. However, I do not yet understand the root cause of the problem, so you might see the problem reemerge again (temporarily) as I make further changes trying to chase it to ground. Thank you for your patience. Do not hesitate to alert me if the problem reappears, or if you find other issues in the new code.

(6) By Aslak Raanes (aslakr) on 2018-07-31 14:18:09 in reply to 1 [link] [source]

This looks like a very nice feature.

Any chance one could show the display name/real name in addition to the shorter username?

(11) By Aslak Raanes (aslakr) on 2018-07-31 14:39:33 in reply to 6 [link] [source]

I mean showing

By Richard Hipp (drh) …

instead of just

By drh

(12) By Dan Barbarito (danbarbarito) on 2018-07-31 14:53:23 in reply to 11 [link] [source]

I like this idea a lot.

(14) By Richard Hipp (drh) on 2018-07-31 14:55:16 in reply to 11 [source]

That seems like a reasonable request. But there are subtle security concerns that I need to think through first....

The username (ex: "drh") is part of the message artifact, and is thus part of the block chain of the repository. But the mapping from username to display name is local repository information, contained in the USER table. So the username is synced but the display name is not. So you could get into a situation where somebody clones a forum repository but then sets up a different USER table providing different names to the posts. I can't really think of scenario where that presents a security problem, but it still gives me pause and makes me want to think about it for a while.

(31) By Aslak Raanes (aslakr) on 2018-07-31 17:09:17 in reply to 14 [link] [source]

Without looking at the code, does this also apply for username to email mapping?

The second thing I was about to suggest for visualising who the poster is by using the email adresse for fetching an avatar, either from gravatar or libravatar (ie. querying DNS for which avatar server to use). Or maybe just make it possible to upload an avatar?

(32) By Aslak Raanes (aslakr) on 2018-07-31 17:15:08 in reply to 11 [link] [source]

Maybe related:

"User Name versus Display Name" https://fossil-scm.org/forumtest1/forumpost/45481cc6d9

(7) By anonymous on 2018-07-31 14:26:09 in reply to 1 [link] [source]

Good work!

  • Is there any way to sort threads? via tags or sections?

  • Is there a simple way to delete messages (spam, troll, etc.)? What happens if we delete the artifact of a message?

  • What about merging or splitting threads?

(This is an anonymous post.)

(9) By Richard Hipp (drh) on 2018-07-31 14:30:53 in reply to 7 [link] [source]

There is a search feature. But there is currently no other way to "sort threads". I'm not exactly sure what that would look like, nor am I clear on what the advantages would be. Can you elaborate?

Spams and trolls can be deleted. You just don't see those buttons because you are not logged in as a user who has the authority to do that kind of thing. :-)

The artifact design includes the thread-id in the artifact. So an artifact is tied to a particular thread when it is created. It would, in theory, be possible to change that using a tag, but it would be hard. So, there is not currently a way to merge or split threads. When does the need for that sort of thing arise?

(13) By anonymous on 2018-07-31 14:55:09 in reply to 9 [link] [source]

anonymous thanks :)

(19) By anonymous on 2018-07-31 15:24:16 in reply to 9 [link] [source]

But there is currently no other way to "sort threads". I'm not exactly sure what that would look like, nor am I clear on what the advantages would be. Can you elaborate?

There is usually two ways of sorting threads:

  • via tags (like on Stackoverflow: tags for each language for instance.

  • via sections (like on PHPBB and many other forums): an example with the OpenOffice forum.

The Discourse forum allows to sort via tags or sections. Example with the Mozilla forum.

Having one way or another to sort threads is very useful to avoid mess and find what we look for.

there is not currently a way to merge or split threads. When does the need for that sort of thing arise?

Merging and splitting are useful to tidy messy discussions.

Unfortunately, it’s not rare that people post on the wrong thread or that an endless discussion is going far away from the initial topic.

Some people post before searching if a topic has already been discussed. Merging threads is useful to link similar topics that are popping up again and again.

(15) By Dan Barbarito (danbarbarito) on 2018-07-31 15:04:51 in reply to 1 [link] [source]

I noticed that the forum search only searches message content. Can we also make it search usernames? If I search "danbarbarito" I would expect all my posts to come up, but currently it shows no results.

(17) By Richard Hipp (drh) on 2018-07-31 15:13:12 in reply to 15 [link] [source]

Please double-check. I thought I had rigged it so that it also searched for the title and the username. But it was late last night when I wrote that code, so there could easily be a bug. If possible, provide a test case to help me track it down.

There are actually two search mechanisms in Fossil. There is grep-like search that actually scans the entire corpus on each search request. Then there is a second system that uses SQLite's FTS4 mechanism. Maybe the search-for-username only works on the simpler grep-like search. This repository is used FTS4.

(21) By Dan Barbarito (danbarbarito) on 2018-07-31 16:02:55 in reply to 17 [link] [source]

Searching by username still does not work for me

(29) By Richard Hipp (drh) on 2018-07-31 16:49:43 in reply to 21 [link] [source]

I have (temporarily) turned off the FTS4 search and so searching has reverted to the less efficient full-corpus scan technique. And with that configuration change, search for usernames appears to be working. Please confirm. I will strive to get FTS4 search working again soon.

(30) By Dan Barbarito (danbarbarito) on 2018-07-31 16:56:03 in reply to 29 [link] [source]

Its working for me now! Thanks a lot.

(16) By Johnn5 (dj_goku) on 2018-07-31 15:07:58 in reply to 1 [link] [source]

This is awesome DRH!

Great work from Fossil-SCM and SQLite! This community rocks!

(18) By Marcelo Huerta (richieadler) on 2018-07-31 15:20:00 in reply to 1 [link] [source]

Is there, or will there be soon, a full documentation of how to configure this new capability in a repository?

(20) By sean (jungleboogie) on 2018-07-31 16:02:41 in reply to 1 [link] [source]

Hi DRH,

When I registered for an account, I wasn't prompted to confirm my email address. I believe when subscribing to the email fossil-src activity alert emails, I needed to confirm my subscription. I might me mistaken, though.

I really like how the threads are laid out, very much like hackernews and lobsters style. If a thread is super long, will it carry on forever or have a 'more' button, similar to the timeline view?

It might be worth talking about wrapping some of the comments, or is that handled in the css skin of what's running the fossil repo?

Keep up the great work!

(22) By Richard Hipp (drh) on 2018-07-31 16:09:54 in reply to 20 [link] [source]

> When I registered for an account, I wasn't prompted to confirm my email address

See https://fossil-scm.org/forumtest1/forumpost/34019bf5b8876e684def512

(23) By mike-test (mike) on 2018-07-31 16:22:18 in reply to 1 [link] [source]

The sheer speed features go through suggestion / discussion / implementation phases in this community is mind-boggling! Thank you!

(27) By sean (jungleboogie) on 2018-07-31 16:36:12 in reply to 23 [link] [source]

yes, I agree! DRH is very clever, generous, and genuine with wanting to make great software.

(24.1) By sean (jungleboogie) on 2018-07-31 16:33:26 edited from 24.0 in reply to 1 [link] [source]

I think the delete button is acting as a reply. thread in question

I can't seem to delete any of those posts that were all created by me, but I can reply to them. The 'delete' button is actually a reply button, but I can't delete any of my replies or original messages.

(28) By Richard Hipp (drh) on 2018-07-31 16:36:28 in reply to 24.1 [link] [source]

The Delete button is suppose to work like an Edit that sets the body of the message to an empty string.

Remember - this is block chain. You cannot remove old records, you can only insert new records that modify the old records. The Delete button works by inserting a new artifact that overrides the text of an older artifact so that the text is now an empty string.

(33) By ㎝ (_cm) on 2018-07-31 17:28:09 in reply to 28 [link] [source]

after you have deleted a post, the [Replay] and [Edit] buttons disappears, but not the [Delete] button; if you press it (in order to Delete the Deleted Post? ;) it opens the reply form.

(25) By sean (jungleboogie) on 2018-07-31 16:26:21 in reply to 1 [link] [source]

I don't see a reply button for Mike's post. Is that because he's anonymous?

(26) By Richard Hipp (drh) on 2018-07-31 16:30:57 in reply to 25 [link] [source]

No, it was because his post was pending moderation. You cannot reply to an unmoderated post. You should be able to reply now, though.

(34) By anonymous on 2018-07-31 17:47:37 in reply to 1 [link] [source]

Would you please consider also emulating the 'digest' feature of the mailing list? It shouldn't be hard, since the emailing functionality is already there. Or maybe I am missing something...

(35) By Dan Barbarito (danbarbarito) on 2018-07-31 18:58:24 in reply to 34 [link] [source]

I believe this functionality is already here. Visit the /alerts page and then select "Daily digest only".

(36) By anonymous on 2018-07-31 20:21:16 in reply to 35 [link] [source]

Wow! :)

(37) By Dingyuan Wang (gumblex) on 2018-07-31 23:38:37 in reply to 1 [link] [source]

I think the post list should show author id in addition to time like other forums.

(39) By Chris (cwr) on 2018-08-01 01:27:08 in reply to 37 [link] [source]

Yes, this would be great to have on the post list page.

Amazing work drh, this is really cool! Thank you!

(markdown formatting for testing purposes too :D )

(40.1) By Martijn Coppoolse (vor0nwe) on 2018-08-01 11:26:04 edited from 40.0 in reply to 37 [link] [source]

I agree!

Also, if at all possible, in the search results. Those currently don’t show the date/time either, which is often helpful in determining the relevancy of the given search result.

Ideally, the search results could be grouped by thread. But that would probably introduce quite a lot of complexity.

(edit: a word)

(38) By Kees Nuyt (knu) on 2018-08-01 00:41:44 in reply to 1 [link] [source]

At this moment, I can't tell whether I like it or not. I have to get used to this.

The mailing list is so convenient, I hope this will work out equally convenient.

Best Regards,

Kees

(45) By Warren Young (wyetr) on 2018-08-01 18:46:35 in reply to 38 [link] [source]

The mailing list is so convenient

Mailing lists are also convenient to spammers. Since drh's time is limited, and it will be far easier to control spammers via this forum, this move means we either get more SQLite and Fossil features in the long run, or drh gets more free time, or some combination of these. I like mailing lists, too, but these seem like good goals to achieve.

Consider also the value of having your own projects' conversation histories backed up in the repo and sync'd to every clone. In this world, past conversations are a quick search away, even while disconnected from the network.

A project's conversation history is roughly as valuable as its wiki articles, embedded documentation, and checkin comments.

Now consider the value in a world where all of that is interlinked: tickets can refer to forum posts, forum posts can refer to wiki articles, etc., all by giving internal links within the same repository. Test.

There's also a lot of value in sane formatting: mailing lists operators have a poor choice between allowing only plain text or allowing HTML email. HTML email is far too much power to allow, but plain text has a bunch of weaknesses we don't need to put up with in 2018. The Markdown and wiki syntaxes fix this, giving us just enough power. We know it's enough because it's been proven out over millions of users and many years in Stack Exchange, Github, and elsewhere.

(47) By sean (jungleboogie) on 2018-08-01 19:02:42 in reply to 45 [link] [source]

I was initially going to reply to Roy as well, but it wouldn't have been succinct as this. Thanks for making these points.

So far the forum has turned out better than I thought it would have, and we know folks will provide patches and suggestions on improvements.

Now that the forum is created, I was thinking of things like this:

1. Code reviews of projects can also be apart of the repo. I think the initial idea for all of this started when folks were stating github allows easier pull requests. Now that the forum is working, it might mean pull requests can happen in a github style.

2. If a bug is raised on a forum post, it would be cool if that could be converted into a bug report on the project's repo with a single button. There might be some clean up in the bug to do, but it could easily cross link bug to forum post, and forum post to bug.

(41) By Martijn Coppoolse (vor0nwe) on 2018-08-01 10:49:43 in reply to 1 [link] [source]

Not sure if I should post this here, or in a separate thread, but...

When registering for alerts, the form won't accept the plus sign (+) on the left of the at sign of my e-mail address:

← illegal character in email address: 0x2b '+'

The plus sign is definitely a legal character at that position.

It's actually fairly common to create separate "sub-addresses" that way. E.g. mail sent to jane.doe+fossil@gmail.com will arrive in the same mailbox as jane.doe@gmail.com. Many mail servers support this. This makes it very easy to filter messages addressed to that specific e-mail address (e.g. to move them to a different folder, or whatever).

See also The correct way to validate email addresses (and the accompanying HackerNews discussion).

(42.1) By sean (jungleboogie) on 2018-08-01 14:39:29 edited from 42.0 in reply to 41 [link] [source]

Hi,

yes, I also reported this on this post.

If you submit a patch, he'll probably take it into consideration!

(43) By sean (jungleboogie) on 2018-08-01 14:41:37 in reply to 1 [link] [source]

When replying to a post, either the top thread post or a subpost, it would be nice to see the username you're replying to.

(44) By JohnQSmith on 2018-08-01 18:14:46 in reply to 1 [link] [source]

Looks awesome. I recompiled a week or two ago after updating to the forum branch, but didn't see (didn't have too much time to actually look for) where to set it up.

(46) By Richard Hipp (drh) on 2018-08-01 19:00:10 in reply to 44 [link] [source]

Documentation is on my to-do list. Let me get it working first....

(48) By JohnQSmith on 2018-08-01 21:32:45 in reply to 46 [link] [source]

No problem whatsoever!

Take your time.

I was just wanting to catch a quick glance at the work in progress.

I did get it to compile on Windows
(after doing an nmake clean)
so it was a worthwhile endeavor nonetheless.

And while this is still testing, I can

  • tinker
  • with
  • markdown
  • support

(49) By stevel on 2018-08-02 08:43:04 in reply to 1 [link] [source]

An observation (not a criticism) is that (at its current state of development) the forum does not work too well on mobile devices. And so IMO it isn’t yet a replacement for the mailing list.

(50) By Richard Hipp (drh) on 2018-08-02 11:13:02 in reply to 49 [link] [source]

I have seen that too, and agree that it is something that needs fixing.

(51) By Dubhead on 2018-08-03 08:58:27 in reply to 1 [link] [source]

> Because each message is an artifact, forum content syncs just like everything else.

Does this mean, instead of subscribing to a mailing list, we do `fossil clone` the forum to perticipate in it, and do `fossil update` to get new posts?

(52) By Richard Hipp (drh) on 2018-08-03 11:38:34 in reply to 51 [link] [source]

Since you are participating the new forum now, without having cloned it, surely this was a facetious questions, right?