Fossil Forum

Fossil JSON API ignoring POST data
Login

Fossil JSON API ignoring POST data

(1) By Iván Ávalos (avalos) on 2020-07-09 01:39:41 [source]

I'm using the latest Fossil (2.12), at least the last in trunk a while ago. I was testing on stephan's Fossil server (sorry, I didn't ask for permission), doing anonymous logins, and it was working fine. He's serving through CGI, and I'm serving through HTTP and NGINX.

HTTP Request

POST /personal-website/json/login HTTP/1.1
Host: fossil.avalos.me
Content-Type: application/json

{
	"name": "avalos",
	"password": "xxxxxxxxx"
}

HTTP Response

{
    "fossil": "dd490d17bec777c4cc26aecfa672d8c29a69292e7c867558023549c11322c553",
    "timestamp": 1594257998,
    "resultCode": "FOSSIL-2103",
    "resultText": "Login failed - password not supplied",
    "command": "login",
    "procTimeUs": 1090,
    "procTimeMs": 1
}

Fossil command

/usr/local/bin/fossil server /var/www/fossil --https --port 8079

NGINX config

location / {
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_read_timeout 90;
    proxy_redirect http://127.0.0.1:8079 $scheme://fossil.avalos.me;
    proxy_pass http://127.0.0.1:8079;
}

NGINX logs

3.13.12.160 - - [09/Jul/2020:01:35:08 +0000] "POST /personal-website/json/login HTTP/1.1" 200 230 "-" "PostmanRuntime/7.24.1"

(2) By Stephan Beal (stephan) on 2020-07-09 01:52:15 [link] [source] in reply to 1

This is very probably related to the previous problem with the path, just in another spot. i'll take a look at it in the morning. By all means, feel free to use one of my repos for comparison.

The issue isn't the NGINX, but is a side effect of serving a directory of fossil files rather than one repo per path/script. That's a feature i have never used, so wasn't aware that doing so changes the request path, which invalidates the "is this a JSON request?" check.

(3) By Stephan Beal (stephan) on 2020-07-09 02:24:09 [link] [source] in reply to 1

doing anonymous logins...

Couldn't sleep (one of the arguable benefits of early retirement is a complete lack of sleep schedule). Please try again with:

https://fossil-scm.org/fossil/info/23e138e808ff3406

That version is now also on my hoster (feel free to use that one for comparison purposes).

Summary: same problem as reported in /forumpost/e4953666d6, just in a different place (/json needs to handle the POST data differently than non-JSON routes do). IIRC those were the only two places where that check was necessary, so that "should" be the last time we see that particular bug.

(4) By Iván Ávalos (avalos) on 2020-07-09 02:42:08 [link] [source] in reply to 3

Thanks! (You should really take care of your sleep schedule, though).

But, the problem persists. :( I tried with your last commit (23e138e808), and still getting the same response from the API:

{
    "fossil": "23e138e808ff34066a91ce1b454b928bb2de898795770a72b001e1a0f84c6a6b",
    "timestamp": 1594262369,
    "resultCode": "FOSSIL-2103",
    "resultText": "Login failed - password not supplied",
    "command": "login",
    "procTimeUs": 1091,
    "procTimeMs": 1
}

(7) By Stephan Beal (stephan) on 2020-07-09 03:26:35 [link] [source] in reply to 4

But, the problem persists.

i'm unfortunately unable to reproduce that. After fixing a bug in fossil.fetch() (just checked in), i opened up the JS console on a directory-mode server instance, visited the /fileedit page (because fossil.fetch() is currently only loaded by that page) and ran:

fossil.fetch('json/login', {payload:{payload:{name:"stephan",password:"xxxxxxxx"}}})

(That outer "payload" property is part of the fetch() interface, the inner one is for the POSTed JSON data.)

That submitted a POST to http://localhost:8080/cwal/json/login (note the "cwal" part - that's from the cwal.fossil file in the served directory) with a body of:

{"payload":{"name":"stephan","password":"xxxxxxxx"}}

And responded with (manually indented for readability):

{"fossil":"23e138e<snip>",
"timestamp":1594263942,
"command":"login",
"procTimeUs":2723,
"procTimeMs":2,
"payload":{
  "authToken":" snip>",
  "name":"stephan",
  "capabilities":"s",
  "loginCookieName":"<snip>"
}}

(Why fossil.fetch()? Solely because it's the only halfway-convenient option within easy reach. It's just an XHR proxy, not doing anything magical.)

Before version 23e138e, it could not have logged me in for that case because the /cwal path prefix would have confused the POST input handler.

i don't currently have a hypothesis about why it's not working for you. A cached response (which "shouldn't" happen for POST) can be ruled out: you're getting the response timestamp in the response.

:-?

i'll try again with the anonymous login soon (first need to go RTFM - it's been many years since i've used JSON to log in anonymously).

(9) By Stephan Beal (stephan) on 2020-07-09 03:49:57 [link] [source] in reply to 7

i'll try again with the anonymous login soon (first need to go RTFM - it's been many years since i've used JSON to log in anonymously).

Anonymous is working for me as well via a directory-mode server:

GET /cwal/json/anonymousSeed

Responds with the anonymous login seed and password, which are then POSTed to /cwal/json/login with a body of:

{"payload":{"anonymousSeed":xxxx,"password":"xxxx","name":"anonymous"}}

and the response is:

{
"fossil":"23e138e...",
"timestamp":1594265856,
"command":"login",
"procTimeUs":1840,
"procTimeMs":1,
"payload":{
  "authToken":"<snip>/anonymous",
  "name":"anonymous",
  "capabilities":"hmnc",
  "loginCookieName":"<snip>"
}}

It's also working on my older CGI-based JSON wikis, e.g.:

https://fossil.wanderinghorse.net/wikis/cson

Try the anonymous login button there. (Logout isn't working property, but that's just a UI update problem. After tapping logout, tap the "Who?" button to reset the login fields.)

Presumably NGINX is forwarding POSTed data? Googling "nginx forward post request" reveals that it apparently needs to be set up to do so, but my experience with NGINX is zero so i can't make any concrete suggestions about how to do/check that.

(10) By Iván Ávalos (avalos) on 2020-07-09 03:56:36 [link] [source] in reply to 9

Nevermind, it's already working! I forgot to set the envelop in the request! But yeah, it's working now, I really appreciate you taking the time to fix all those things, I'm looking forward to contribute directly to Fossil's source code directly soon. Thanks very much!

(11) By Stephan Beal (stephan) on 2020-07-09 04:04:59 [link] [source] in reply to 10

Nevermind, it's already working! I forgot to set the envelop in the request!

FWIW, i did that on my first test, too ;). The JSON interface is admittedly a bit over-engineered in that regard :/.

Thanks very much!

And thank you for the bug reports. That specific use case has been broken for years, but nobody noticed.

(5) By Iván Ávalos (avalos) on 2020-07-09 02:43:57 [link] [source] in reply to 3

By the way, the code is pretty straightforward to read, even if it is plain C. I'll try to give it a read and see if I can help to find something! :)

(6) By Iván Ávalos (avalos) on 2020-07-09 02:57:52 [link] [source] in reply to 3

Does src/cgi.c affect the handling of HTTP requests via fossil server?

(8) By Stephan Beal (stephan) on 2020-07-09 03:28:55 [link] [source] in reply to 6

Does src/cgi.c affect the handling of HTTP requests via fossil server?

Yes, at least for any parts relevant to this thread.