Fossil Forum

fossil chat pull - requires password to be remembered
Login

fossil chat pull - requires password to be remembered

fossil chat pull - requires password to be remembered

(1) By mxb on 2024-02-22 14:44:23 [source]

I just tried
fossil chat pull
with fossil 2.23 compiled from source code for a Lubuntu-18.04 32-Bit system.

Without a remembered password, it did not ask for a password for the remote connection
and I got the following response:

SQLITE_ERROR(1): no such table: chatbu.chat in "SELECT count() FROM chatbu.chat" Database error: no such table: chatbu.chat SELECT count() FROM chatbu.chat

fossil chat pull --debug would give an output like this:

REQUEST_URI: <path_to_repo_on_server>/chat-backup?msgid=<msgid>&resid=<resid> URL: https://<user>@<domain>/path/to/repo Sending 169 byte header and 0 byte payload Read: [HTTP/1.1 200 OK] Read: [Date: Thu, 22 Feb 2024 14:33:31 GMT] Read: [Server: Apache/2.4.57 (Unix) OpenSSL/1.0.2u] Read: [Cache-control: no-cache] Read: [X-Frame-Options: SAMEORIGIN] Read: [Content-Length: 0] Read: [Connection: close] Read: [Content-Type: text/html; charset=utf-8] SQLITE_ERROR(1): no such table: chatbu.chat in "SELECT count() FROM chatbu.chat" Database error: no such table: chatbu.chat SELECT count() FROM chatbu.chat

fossil chat pull
however does work for me, if I let the remote password be remembered.

(2.1) By Stephan Beal (stephan) on 2024-02-22 17:00:00 edited from 2.0 in reply to 1 [link] [source]

however does work for me, if I let the remote password be remembered.

Thank you for the report and reproduction steps. It's reproducible here and is being looked into.

It fails in the same way, even with a stored password, over an HTTP (non-HTTPS) connection:

$ f remote-url http://stephan@localhost:8081/cgi-bin/fossil -R f.fossil
password for http://stephan@localhost:8081/cgi-bin/fossil: 
remember password (Y/n)? Y

$ f chat pull --unsafe --debug -R f.fossil
REQUEST_URI: /cgi-bin/fossil/chat-backup?msgid=0&<snip>
URL: http://stephan@localhost:8081/cgi-bin/fossil
Sending 184 byte header and 0 byte payload
Read: [HTTP/1.0 200 OK]
Read: [Cache-control: no-cache]
Read: [X-Frame-Options: SAMEORIGIN]
Read: [Content-Type: text/html; charset=utf-8]
Read: [Content-length: 0]
SQLITE_ERROR(1): no such table: chatbu.chat in "SELECT count(*) FROM chatbu.chat"
Database error: no such table: chatbu.chat
SELECT count(*) FROM chatbu.chat

Edit: an obscure()d password was inadvertently posted above (now removed), but that password isn't in use on any repos outside of my LAN, so no harm done.

(3.2) By mxb on 2024-02-22 16:50:57 edited from 3.1 in reply to 2.0 [link] [source]

It fails in the same way, even with a stored password, over an HTTP (non-HTTPS) connection:

That is interesting.

I thought, it was only connected to the fact that
fossil chat pull
did not ask for a password, when I assumed it should.

"Things are never quite the way they seem." ;)

It is also interesting, if there should be a table "chatbu.chat"
or if that string/name somehow occurred by accident...

I have not found such a table before or after successfully pulling chat content.

The remote server runs Fossil-2.22 in my case.

(4) By Stephan Beal (stephan) on 2024-02-22 17:11:38 in reply to 3.2 [link] [source]

did not ask for a password, when I assumed it should.

FWIW, that was also my hope ;), but...

"Things are never quite the way they seem." ;)

Indeed! That still seems to be the underlying problem but i've not yet identified where it's (not) happening (and may not today, but am looking into it).

It is also interesting, if there should be a table "chatbu.chat"

"chatbu" is the schema name under which the db gets ATTACHed for purposes of importing the chat on the local client. You won't find that schema on the remote. The backup is written into a transient in-memory db, gets gets ATTACHed with the name "chatbu".

What's happening is that the chat pull appears to take place, but no data is actually transferred from the remote. The chat pull command then tries to ATTACH the incoming zero-byte db and copy those records over into the local repo. It fails then fails because it can't read a "chat" table from the ATTACHed (empty) db.

You can also repro this, with slightly different end results, using:

$ fossil chat pull --debug -R REPO --out mychat.db

the intended result is that mychat.db holds the results, but it's 0 bytes in the case you've found.

(5) By mxb on 2024-02-22 17:51:13 in reply to 4 [link] [source]

Ah, yes.
Now I understand about that "chatbu.chat".

the intended result is that mychat.db holds the results, but it's 0 bytes in the case you've found.

I had also tried it with an --out database.
And indeed, it had reported 0 bytes of size for that new database when it failed.

Meanwhile I had successfully fetched some chat contents to the cloned local repo.

When I then tried to get those contents pulled to that extra database,
that size went up to 1024 bytes.
Apparently the chat table had been created in that database,
but I still had not gotten any contents there.

So, I learnt, that that is apparently the occasion to use the --all option.

Doing that, did then indeed get me the contents to that extra database.

(7) By Stephan Beal (stephan) on 2024-02-22 18:16:30 in reply to 5 [link] [source]

When I then tried to get those contents pulled to that extra database, that size went up to 1024 bytes. Apparently the chat table had been created in that database,

Correct, but it does not contain any content in the 1kb-db case because there are no new messages to pull.

Try this:

$ fossil sqlite -R YOUR-REPO
...
delete from chat;
.quit
$ fossil chat pull --debug -R YOUR-REPO

It should pull over all remote messages because your local repo db has none. Then try:

$ fossil chat pull --debug --out foo.db -R YOUR-REPO

What will happen is foo.db will be created (with a size of 1kb) but will have no data because there was nothing new to pull.

(10) By mxb on 2024-02-22 22:51:54 in reply to 7 [link] [source]

I had used fossil sql only in non interactive mode,
with sql command strings given in the command line, up to today.

I had not even known, it could also offer an interactive mode.

Of course I could have opened the repo with sqlite to get some interactive mode...

I am very impressed about those possibilities new to me.
It appears to have some nice features.

And there are all the relevant databases available already:

  • the repository,
  • the checkout database and
  • the global config database.

:)

I have tried a few things with

fossil chat pull ...

Apparently, I only get to see, how many new records I get, when pulling to the cloned repo.

$ fossil chat pull Got 2 new records, 1024 bytes

When pulling them to an extra database (--out Database), I do not get to see, how many new records I get.

$ fossil chat pull --out ../chat2_out.sqlite Chat database at ../chat2_out.sqlite is 1024 bytes

I think, that is a problem, especially since it is possible, that I do not get any new records for various reasons:

  • if I have posted messages in the chat in the local clone, I may not get any records from the remote repo, whether I try to pull it to that repo or an external db (perhaps depending on the timestamps of the messages...)
  • if I have already pulled the chat to the local clone, I would not get those messages to the extra db any more either.

Of course, using the option --all, will fix that problem.

That way, it appears to merge the messages from the remote repo with those in the local repo,
while it appears to replace the messages in the extra database with those from the remote repo.

Considering, that pulling the chat data may be intended to keep a record of as many messages as possible,
that merging may be the better option, as it may keep even messages, meanwhile deleted in the remote repo. (?)
At least, I can't think of a good reason to treat the local clone differently than the extra database at the moment.

Disclaimer: I may have been somewhat superficial, especially with the last few tryouts, so a few (hopefully minor) errors or misinterpretations are possible.

(6) By Stephan Beal (stephan) on 2024-02-22 18:09:07 in reply to 1 [link] [source]

however does work for me, if I let the remote password be remembered.

This patch seems to resolve it but has not yet been extensively tested to rule out unwanted side effects:

Index: src/chat.c
==================================================================
--- src/chat.c
+++ src/chat.c
@@ -1217,11 +1217,19 @@
       zObs = obscure(g.url.user);
       blob_appendf(&reqUri, "&resid=%t", zObs);
       fossil_free(zObs);
     }
     zPw = g.url.passwd;
-    if( zPw==0 && isDefaultUrl ) zPw = unobscure(db_get("last-sync-pw", 0));
+    if( zPw==0 && isDefaultUrl ){
+      zPw = unobscure(db_get("last-sync-pw", 0));
+      if( zPw==0 ){
+        /* Can happen if "remember password" is not used. */
+        g.url.flags |= URL_PROMPT_PW;
+        url_prompt_for_password();
+        zPw = g.url.passwd;
+      }
+    }
     if( zPw && zPw[0] ){
       zObs = obscure(zPw);
       blob_appendf(&reqUri, "&token=%t", zObs);
       fossil_free(zObs);
     }

That will simply prompt for the password if it cannot otherwise find one. It does not offer to save the newly-entered password, the thinking being that the user is in this situation because they very specifically declined to save the password before and presumably want to do the same this time.

Are you able to apply that change and confirm whether or not it resolves the problem for you?

Curiously, this does not fix the failure of it not fetching the chat messages over http (as opposed to https). That failure is still happening regardless of whether the password is remembered/provided or not, so it apparently a different problem.

(8) By mxb on 2024-02-22 18:38:16 in reply to 6 [link] [source]

Are you able to apply that change and confirm whether or not it resolves the problem for you?

Basically yes, but it will take some time, until I can get to try it.

(9) By mxb on 2024-02-22 21:24:59 in reply to 6 [link] [source]

Are you able to apply that change and confirm whether or not it resolves the problem for you?

It works as described.
It asks for a password, but does not ask to store it.

I like that behavior in that context.

Of course, I like the option to change my mind (about the password being remembered or not), but I like as little repeated questions as possible.

The question about remembering the password does reoccur at least when doing sync or pull...
at least how I used that new remote repo so far.

When I want that remembered password to be forgotten,
I can of course do:

fossil remote scrub