Cause of this Problem

I'm on OpenBSD-6.7-current. (2020-Aug-14) (2020-Aug-18)

Update 1: tested on 2.12 (same behaviour)

Update 2: no use of gimmicky solution as all POST request fails more

$ fossil version
This is fossil version 2.11.2 [c58877d6f2] 2020-08-20 13:24:41 UTC
This is fossil version 2.12.1 [b98ce23d4f] 2020-08-20 13:27:04 UTC

Following the instructions for fastcgi on httpd doc, the directory page lists all *.fossil files, but opening a particular repository results in

Bad Request: missing SCRIPT_NAME

not a solution (dirty trick that works on firefox), adding a SCRIPT_NAME parameter to fastcgi protocol in httpd.conf

location  "/*" {
    fastcgi { 
        param SCRIPT_FILENAME "/cgi-bin/scm"
        param SCRIPT_NAME " "
    }

I'm unsure of this behavior, can anyone replicate and confirm this.

I have re-tested 2.10 on this same box, working fine without any of these issues.

more info,

This is fossil version 2.11.1 [307d94c549] 2020-06-08 19:48:29 UTC
Compiled on Aug  9 2020 20:14:54 using clang-10.0.1  (64-bit)
Schema version 2015-01-24
Detected memory page size is 4096 bytes
zlib 1.2.3, loaded 1.2.3
hardened-SHA1 by Marc Stevens and Dan Shumow
SSL (LibreSSL 3.2.1)
libfuse 2.6, loaded unknown
FOSSIL_ENABLE_LEGACY_MV_RM
UNICODE_COMMAND_LINE
FOSSIL_STATIC_BUILD
HAVE_PLEDGE
SQLite 3.32.1 2020-05-25 16:19:56 0c1fcf4711
SQLITE_DEFAULT_FILE_FORMAT=4
SQLITE_DEFAULT_WAL_SYNCHRONOUS=1
SQLITE_ENABLE_DBSTAT_VTAB
SQLITE_ENABLE_FTS4
SQLITE_ENABLE_FTS5
SQLITE_ENABLE_JSON1
SQLITE_ENABLE_LOCKING_STYLE=0
SQLITE_ENABLE_STMTVTAB
SQLITE_LIKE_DOESNT_MATCH_BLOBS
SQLITE_MAX_EXPR_DEPTH=0
SQLITE_OMIT_DECLTYPE
SQLITE_OMIT_DEPRECATED
SQLITE_OMIT_GET_TABLE
SQLITE_OMIT_LOAD_EXTENSION
SQLITE_OMIT_PROGRESS_CALLBACK
SQLITE_OMIT_SHARED_CACHE
SQLITE_THREADSAFE=0
SQLITE_USE_ALLOCA

This is fossil version 2.12 [d075b6199c] 2020-08-16 13:18:06 UTC
Compiled on Aug 17 2020 17:33:46 using clang-10.0.1  (64-bit)
Schema version 2015-01-24
Detected memory page size is 4096 bytes
zlib 1.2.3, loaded 1.2.3
hardened-SHA1 by Marc Stevens and Dan Shumow
SSL (LibreSSL 3.2.1)
libfuse 2.6, loaded unknown
FOSSIL_ENABLE_LEGACY_MV_RM
UNICODE_COMMAND_LINE
FOSSIL_STATIC_BUILD
HAVE_PLEDGE
SQLite 3.33.0 2020-08-14 13:23:32 fca8dc8b57
SQLITE_DEFAULT_FILE_FORMAT=4
SQLITE_DEFAULT_WAL_SYNCHRONOUS=1
SQLITE_ENABLE_DBSTAT_VTAB
SQLITE_ENABLE_FTS4
SQLITE_ENABLE_FTS5
SQLITE_ENABLE_JSON1
SQLITE_ENABLE_LOCKING_STYLE=0
SQLITE_ENABLE_STMTVTAB
SQLITE_LIKE_DOESNT_MATCH_BLOBS
SQLITE_MAX_EXPR_DEPTH=0
SQLITE_OMIT_DECLTYPE
SQLITE_OMIT_DEPRECATED
SQLITE_OMIT_LOAD_EXTENSION
SQLITE_OMIT_PROGRESS_CALLBACK
SQLITE_OMIT_SHARED_CACHE
SQLITE_THREADSAFE=0
SQLITE_USE_ALLOCA

cgi-debug log

-------- BEGIN cgi at 2020-07-16 15:58:43 --------
DOCUMENT_ROOT = /htdocs/fsl.displ.nl
GATEWAY_INTERFACE = CGI/1.1
HTTP_ACCEPT = text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
HTTP_ACCEPT_ENCODING = gzip, deflate, br
HTTP_ACCEPT_LANGUAGE = en-US,en;q=0.5
HTTP_CONNECTION = keep-alive
HTTP_HOST = fsl.displ.nl
HTTP_REFERER = https://fsl.displ.nl/
HTTP_USER_AGENT = Mozilla/5.0 (X11; Linux x86_64; rv:79.0) Gecko/20100101 Firefox/79.0
PATH_INFO = /badges/home
REMOTE_ADDR = 10.0.0.5
REMOTE_PORT = 33892
REQUEST_METHOD = GET
REQUEST_URI = /badges/home
SCRIPT_FILENAME = /cgi-bin/scm
SERVER_PROTOCOL = HTTP/1.1
mem-match [REQUEST_URI] = [/badges/home]
no-match [SCRIPT_NAME]
mem-match [PATH_INFO] = [/badges/home]
mem-match [HTTP_ACCEPT_ENCODING] = [gzip, deflate, br]
mem-match [REQUEST_METHOD] = [GET]
-------- END cgi ---------

I think I might have spoken to you elsewhere or at least someone with
the same problem. I can't replicate this on 6.6- or 6.7-release, which
made me question whether something changed in the current codebase but
looking at cvsweb I couldn't find anything.

just in case, can you confirm after running

$ fossil all rebuild

(side note: I was the same person, caught with a httpd bug so couldn't reach eariler)

The same issue has been reported for Apache+Fastcgi,
I couldn't find any update regarding that.

So, I was able to reproduce this: after speaking to you previously at my
blog, I based my reponse on my two existing installations, which are
both on release (6.6 and 6.7). Then after you posted here, I spun up a
new vm with vmd but I was following the instructions verbatim from the
doc. On both 6.6- and 6.7-release, using pkg_add, this installs
fossil version 2.10 [9d9ef82234] 2019-10-04 21:41:13 UTC,
which does not produce this problem. I later realised this wasn't the
same version as your bug report. So I cloned trunk, downloaded the
fossil version 2.11.1 [307d94c549] 2020-06-08 19:48:29 UTC
source, compiled them—and both produce the same problem. In both cases,
however, the proposed fix didn't work and I've yet to find a solution.
To be precise, the fix did remove the missing SCRIPT_NAME error but it
malforms the request to an invalid URI. I'm knee deep in an ML project
at the moment so haven't had the time to investigate any further.

ETA: I've found that it's coming from the if(zScriptName==0)
check in the cgi_init() function from cgi.c; strangely, this is in
the versions that do and do not reproduce this bug.

version 2.12 landed on ports, I'll update quirks and test in a moment.

Fossil needs a few specific CGI parameters. Consider the following example that uses the URL for your previous post:

    https://www.fossil-scm.org/forum/info/12736b30c072551a?t=c
            \________________/\____/\____________________/ \_/
                    |            |             |            |
               HTTP_HOST         |        PATH_INFO     QUERY_STRING
                            SCRIPT_NAME

The CGI script in this case is the file named "forum" in the DOCUMENT_ROOT of the web-server. The name of this file should appear in SCRIPT_NAME. Excess URL components beyond the SCRIPT_NAME are passed through as the PATH_INFO. Fossil needs both of these values in order to compute correct hyperlinks.

If your web server is not supplying SCRIPT_NAME, then how is Fossil suppose to figure out the URL that invoked it? What CGI parameter does your system provide that we might substitute in place of SCRIPT_NAME?

If you want to see the specific values for CGI parameters that are provided by the web server where this forum is running, visit (for example): https://fossil-scm.org/forum/test_env/extra/stuff?a=5&b=6

The Problem isn't Slowcgi (fastcgi protocol used by httpd) providing SCRIPT_NAME, it indeed does

 uid=67, gid=67
g.zBaseURL = https://fsl.displ.nl /badges
g.zHttpsURL = https://fsl.displ.nl /badges
g.zTop = /badges
g.zPath = test_env
g.userUid = 1
g.zLogin = hisacro
g.isHuman = 1
capabilities = abcefghijklmnopqrstwz234567AD
anonymous-adds =
g.zRepositoryName = /htdocs/fsl.displ.nl/badges.fossil
load_average() = 0.000001
cgi_csrf_safe(0) = 0
fossil_exe_id() = 2e1e9b4bee6f8967b11a16f275cd6914
DOCUMENT_ROOT = /htdocs/fsl.displ.nl
GATEWAY_INTERFACE = CGI/1.1
HTTPS = on
HTTP_ACCEPT = text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
HTTP_ACCEPT_ENCODING = gzip, deflate, br
HTTP_ACCEPT_LANGUAGE = en-US,en;q=0.5
HTTP_CONNECTION = keep-alive
HTTP_HOST = fsl.displ.nl
HTTP_REFERER = https://fsl.displ.nl/badges/home
HTTP_USER_AGENT = Mozilla/5.0 (X11; Linux x86_64; rv:76.0) Gecko/20100101 Firefox/76.0
PATH_INFO = test_env
REMOTE_ADDR = 10.0.0.5
REMOTE_PORT = 40158
REQUEST_METHOD = GET
REQUEST_URI = /badges/test_env
SCRIPT_FILENAME = /cgi-bin/scm
SCRIPT_NAME = /badges
SERVER_PROTOCOL = HTTP/1.1
fossil_display_settings = n=50,ss=j,advm=0,y=ci 

as you can SCRIPT_NAME isn't empty, (also note whitespace on zBaseURL, zHttpsURL just for testing I used SCRIPT_NAME as " " otherwise it reports Bad Request)

Problem here is initially leaving SCRIPT_NAME as empty string

This commit? creating problem on httpd+fastcgi (someone previously reported the same with Apache+fastcgi)

So if you back out that change, everything works for you?

Does backing out that change break IIS support? Does anybody know? Is there anybody reading this who has access to an IIS server who can try that out for us?

I'm a little skeptical, still, that SCRIPT_NAME can be an empty string. Unless an entire domain is dedicated to serving a single Fossil repository, the SCRIPT_NAME is needed to distinguish between the various Fossil repositories, and/or other miscellaneous files, that the domain provides. I'm also seeing the "/badge" element of your URL, which is not a Fossil web page and sounds more like the name of a particular repository.

Questions:

• How many resources are you trying to serve from the fsl.displ.nl domain? Do you have just one Fossil repository that answer to that domain and nothing else? If so, what is the "/badge" thing about?

• I really need to see a mapping from input URL (what you type into the browser URL bar) and the CGI parameters. Can you please provide both in the same post?

Backing that change, everything works

(on 2.11.2)

1254c1254
-    if( zValue && zValue[0] ){
---
+   if( zValue ){

(on 2.12.1)

1267c1267
-   if( zValue && zValue[0] ){
---
+    if( zValue ){

(parameters in httpd)

        location  "/*" {
                   fastcgi {
                        param SCRIPT_FILENAME "/cgi-bin/scm" 
                   }
         }

(scm script)

#!/bin/fossil
directory: /htdocs/fsl.displ.nl
notfound: https://displ.nl
repolist
errorlog: /logs/fossil.log

How many resources are you trying to serve from the fsl.displ.nl domain? Do you have just one Fossil repository that answer to that domain and nothing else? If so, what is the "/badge" thing about?

I'm trying to server more than on repository, the root directory of cgi-script has all *.fossil files.

I really need to see a mapping from input URL (what you type into the browser URL bar) and the CGI parameters. Can you please provide both in the same post?

https://fsl.displ.nl/ lists all repos and https://fsl.displ.nl/badges/home goto repo home page.

I don't understand your web server or how it works, but it appears that you are trying to configure your web server to use the "FastCGI" protocol. Fossil does not support FastCGI. So I do not understand how that is working at all. Do you have some other software running that is transforming FastCGI into CGI?

Do you have some other software running that is transforming FastCGI into CGI

SlowCGI provides FastCGI protocol for OpenBSD

For more on this setup doc

So if you back out that change, everything works for you?

I believe I've run into this problem, too. Here are some more details that may help:

• Bisect blames this commit as the source of the problem.

• Both the prior "good" commit and fossil merge --backout d6f69343ca4673d5 atop tip run as expected.

• I'm using my HTTPS + nginx via SCGI on Debian configuration on this server.

• The symptom is infinite 302 redirects until the browser gives up and stops following the redirect.

• This doesn't appear to be the "HTTP → HTTPS" reverse proxy infinite redirect problem again: adding --https to the fossil server --scgi command doesn't help, and redirect-to-https is off anyway.

I'm a little skeptical, still, that SCRIPT_NAME can be an empty string.

It definitely shouldn't be in my configuration since that variable is set per-redirect in the nginx config. It has to be, since the proxying scheme prevents Fossil from handling URLs properly otherwise.

The nginx configuration hasn't changed materially in years. The only actual changes are to add more repos, not to fiddle with working ones, and this commit breaks all repos across the years of operation, not just an identifiable subset.

How many resources are you trying to serve

My current public repo count is 7.

Can you provide:

• The original URL
• The URL that is being redirected to
• Any available details on the specifics of the configuration
• Values of CGI variables

The original URL

There are a bunch, but one is https://tangentsoft.com/pidp8i/, which redirects to the home page, https://tangentsoft.com/pidp8i/wiki?name=Home. The same problem happens with other Fossil pages, not just wiki pages or the home page.

The top-level tangentsoft.com site is static, and the top-level directory maps to the repo name. Everything under that directory is sent through to the Fossil instance serving it.

(This isn't Fossil's built-in "directory" service mode; each repo has a separate Fossil instance running in the background in fossil serve --scgi mode.)

SCRIPT_NAME is set to /pidp8i to achieve all of this.

The URL that is being redirected to

It's an endless loop: Fossil is telling the browser to go back to the same URL it requested each time. The only exception is when you go to the top-level URL and get sent to the home page.

Any available details on the specifics of the configuration

The site configuration is fully documented in the docs I linked to previously, other than some basic anonymization. (Paths, URLs, IPs, user names...)

Values of CGI variables

For the most part, nginx's stock variables are passed straight thru without modification. On the last-working version, /test_env says:

CONTENT_LENGTH = 0
CONTENT_TYPE =
DOCUMENT_ROOT = /usr/share/nginx/html
DOCUMENT_URI = /pidp8i/test_env
FOSSIL_USER = tangent
HOME = /home/tangent
HTTPS = on
HTTP_ACCEPT = text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
HTTP_ACCEPT_ENCODING = gzip, deflate, br
HTTP_ACCEPT_LANGUAGE = en-US,en;q=0.5
HTTP_CONNECTION = keep-alive
HTTP_HOST = tangentsoft.com
HTTP_REFERER = https://tangentsoft.com/pidp8i/stat
HTTP_UPGRADE_INSECURE_REQUESTS = 1
HTTP_USER_AGENT = Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:79.0) Gecko/20100101 Firefox/79.0
PATH_INFO = /test_env
QUERY_STRING =
REMOTE_ADDR = 1.2.3.4
REMOTE_PORT = 64303
REQUEST_METHOD = GET
REQUEST_SCHEME = https
REQUEST_URI = /pidp8i/test_env
SCGI = 1
SCRIPT_NAME = /pidp8i
SERVER_NAME = tangentsoft.com
SERVER_PORT = 443
SERVER_PROTOCOL = HTTP/1.1
USER = tangent
fossil_display_settings = n=50,ss=x,advm=0,y=all 

I am sorry, but I do not know what is going wrong. And without access to your system, there isn't much more I can do to debug it.

We know that backing out check-in d6f69343ca4673d5 breaks CGI for hisacro and leaving it in breaks CGI for you. It would be good if someone could figure out why for both of these, and suggest a proper fix that works for everybody.

Regarding check-in d6f69343ca4673d5, notice that the branch on line 1090 is never taken, because the value of zScriptName is never NULL due to the new default value given on the cgi_parameter() call on line 1076. That can be reasonably called a bug. The changes to the default values on lines 1076 and 1077 seem incorrect. But as those values are all defined on your system, I do not see how it might make a difference to you.

The only other code change is on line 1293 where it goes back to the long-standing behavior of recording the values of environment variables whose value is an empty string. But, again, this seems to not apply in your case.

I will fix the initialization problem on lines 1076 and 1077. But I have no ideas of what to do to get your system working again.

That actually seems to have solved it; my site is running unpatched tip now.

Thanks!

What is the URL that you entered that generates the following (slightly edited) debug log?

-------- BEGIN cgi at 2020-07-16 15:58:43 --------
HTTP_HOST = fsl.displ.nl
PATH_INFO = /badges/home
REMOTE_ADDR = 10.0.0.5
REMOTE_PORT = 33892
REQUEST_METHOD = GET
REQUEST_URI = /badges/home
SCRIPT_FILENAME = /cgi-bin/scm
SERVER_PROTOCOL = HTTP/1.1
-------- END cgi ---------

If the url was "https://fsl.displ.nl/padge/home" then your web server is not working correctly, as it should not be including the "/badge" as part of the PATH_INFO. The PATH_INFO should be "/home" and the SCRIPT_NAME should be "/badge" - assuming that I guessed the URL correct and that "badge" is the name of the CGI.

Yeah, Request-URI is https://fsl.displ.nl/*adges/home,

From the description

Define REQUEST_URI the same as PATH_INFO and
redefine PATH_INFO with SCRIPT_NAME removed from the beginning

Redefining isn't happening because SCRIPT_NAME is left empty.

Is that really an asterisk in the URL?

So, you want to some how configure your web server so that it generates CGI parameters like this:

    https://fsl.displ.nl/badges/home
            \__________/\_____/\___/
                 |         |     |
           HTTP_HOST       |   PATH_INFO
                           |
                      SCRIPT_NAME

Anything else is incorrect. The check-in you identify that ignores blank CGI parameters should not come up here because none of your CGI parameters should be blank.