Fossil

View Ticket
Login
Ticket Hash: 3f0216560679fd41b6afcc84e7b9ffe0dd971fa8
Title: i18n: fossil ui can't show artifacts with non-ascii chars in the name on Windows
Status: Fixed Type: Code_Defect
Severity: Important Priority:
Subsystem: Resolution: Fixed
Last Modified: 2011-05-28 18:50:35
Version Found In: 474850cff5] 2011-05-23 15:11:12
Description:
To reproduce the problem, run the following commands:
<verbatim>
V:\dvcs>fossil init repo
project-id: ca24df3673e790aca6e2d5c361ea1e14d8005d2f
server-id:  b99fe2b3d7c70621cdde33264111135fbd51cc58
admin-user: Tsul (initial password is "47364c")

V:\dvcs>fossil open repo

V:\dvcs>echo 111 > фыва

V:\dvcs>fossil add фыва
ADDED  фыва

V:\dvcs>fossil status
repository:   V:/dvcs/repo
local-root:   V:/dvcs/
server-code:  b99fe2b3d7c70621cdde33264111135fbd51cc58
checkout:     75b33a21e653caa8a18f23ea199cc9c00de6c900 2011-05-26 05:38:22 UTC
tags:         trunk
comment:      initial empty check-in (user: Tsul)
ADDED      ╤Д╤Л╨▓╨░

V:\dvcs>fossil ci -m test
New_Version: 75b33a21e653caa8a18f23ea199cc9c00de6c900

V:\dvcs>fossil close

V:\dvcs>fossil ui repo
Listening for HTTP requests on TCP port 8081
Launch webbrowser: start http://127.0.0.1:8081/
Type Ctrl-C to stop the HTTP server
</verbatim>
Then go to the <i>Files</i> in web-ui and click on the file <i>фыва</i>.

Windows raises a window that "This program has performed an illegal operation and will be closed" a.k.a. general protection fault. When I close this window, it's raised again (and again).

Also this is strange in console, that <code>fossil add</code> shows right file name, but <code>fossil status</code> not.

<hr /><i>anonymous added on 2011-05-26 17:23:52 UTC:</i><br />
I've tried fossil version <tt>[5a4dc2239b] 2011-05-26 11:57:14</tt> on the same repo as mentioned before.<br>
Now <code>fossil status</code> works as expected:
<verbatim>
V:\dvcs>fossil status
repository:   V:/dvcs/repo
local-root:   V:/dvcs/
server-code:  03df0882a9037a202acd1e429d84c98c8d7582a4
checkout:     f26b10b36eba2d83500e430df0ec125df49b4a55 2011-05-26 16:27:48 UTC
tags:         trunk
comment:      initial empty check-in (user: Tsul)
ADDED      фыва
</verbatim>

But <code>fossil ui</code> fails as before. Trace output doesn't give much:
<verbatim>
V:\dvcs>fossil --systemtrace --sqltrace --httptrace ui
SELECT value FROM vvar WHERE name='repository';
ATTACH DATABASE 'V:/dvcs/repo' AS repository;
SELECT 1 FROM config WHERE name='aux-schema'   AND value<>'2011-04-25 19:50';
SELECT value FROM config WHERE name='http-port';
SELECT value FROM config WHERE name='web-browser';
-- LOOKASIDE_USED                 16         48
-- LOOKASIDE_HIT                            184
-- LOOKASIDE_MISS_SIZE                       95
-- LOOKASIDE_MISS_FULL                        0
-- CACHE_USED                  16792
-- SCHEMA_USED                 18688
-- STMT_USED                       0
-- MEMORY_USED                142528     148336
-- MALLOC_SIZE                            48000
-- MALLOC_COUNT                  598        603
-- PCACHE_OVFLOW               19312      19312
-- prepared statements             7
Listening for HTTP requests on TCP port 8080
Launch webbrowser: start http://127.0.0.1:8080/
SYSTEM: "start http://127.0.0.1:8080/"
Type Ctrl-C to stop the HTTP server
SYSTEM: ""V:\dvcs\fossil.exe" http "V:/dvcs/repo" fossil_server_P8080__in1.txt fossil_server_P8080__out1.txt 127.0.0.1 --nossl --localauth"
SYSTEM: ""V:\dvcs\fossil.exe" http "V:/dvcs/repo" fossil_server_P8080__in2.txt fossil_server_P8080__out2.txt 127.0.0.1 --nossl --localauth"
SYSTEM: ""V:\dvcs\fossil.exe" http "V:/dvcs/repo" fossil_server_P8080__in3.txt fossil_server_P8080__out3.txt 127.0.0.1 --nossl --localauth"
SYSTEM: ""V:\dvcs\fossil.exe" http "V:/dvcs/repo" fossil_server_P8080__in4.txt fossil_server_P8080__out4.txt 127.0.0.1 --nossl --localauth"
SYSTEM: ""V:\dvcs\fossil.exe" http "V:/dvcs/repo" fossil_server_P8080__in5.txt fossil_server_P8080__out5.txt 127.0.0.1 --nossl --localauth"
SYSTEM: ""V:\dvcs\fossil.exe" http "V:/dvcs/repo" fossil_server_P8080__in6.txt fossil_server_P8080__out6.txt 127.0.0.1 --nossl --localauth"
^C^C
</verbatim>
Fossil created files with names <tt>fossil_server_P8080__(in|out){4-6}.txt</tt>. All in-files consist of this text:
<verbatim>
GET /artifact?name=bdbdc5bc614b3003e6e5a275ea8482a9b9059698 HTTP/1.1
User-Agent: Opera/9.80 (Windows NT 5.1; U; en) Presto/2.8.131 Version/11.11
Host: 127.0.0.1:8080
Accept: text/html, application/xml;q=0.9, application/xhtml+xml, image/png, image/webp, image/jpeg, image/gif, image/x-xbitmap, */*;q=0.1
Accept-Language: ru-RU,ru;q=0.9,en;q=0.8
Accept-Encoding: gzip, deflate
Referer: http://127.0.0.1:8080/dir?ci=tip
Connection: Keep-Alive

</verbatim>
Out-files are all of the zero length.

What else is that while debugging in Visual Studio it shows that
<blockquote>Unhandled exception at 0x77c37742 in fossil.exe: 0xC0000005: Access violation reading location 0x6c707061.</blockquote>
Stacktrace:
<verbatim>
>	msvcrt.dll!_strcmp()  + 0x12 bytes	
 	fossil.exe!0044461a() 	
</verbatim>
Disassembly:
<verbatim>
	_strcmp:
	77C37730  mov         edx,dword ptr [esp+4] 
	77C37734  mov         ecx,dword ptr [esp+8] 
	77C37738  test        edx,3 
	77C3773E  jne         _strcmp+4Ch (77C3777Ch) 
	77C37740  mov         eax,dword ptr [edx] 
>	77C37742  cmp         al,byte ptr [ecx] 
	77C37744  jne         _strcmp+44h (77C37774h) 
	77C37746  or          al,al 
	77C37748  je          _strcmp+40h (77C37770h) 
</verbatim>
That shows that the exception raises inside msvcrt.dll!_strcmp() function.

I would say more accurate if I had fossil debug symbols or if I was able to compile fossil with Visaul Studio.

<hr /><i>anonymous claiming to be tsul added on 2011-05-27 03:44:12 UTC:</i><br />
First, by default the fossil executable is compiled with <tt>-Os</tt> MinGW gcc option on Windows as I understand. And it fails on conditions described above.

But without <tt>-Os</tt> option it works fine.

Second, here's my debug output of fossil-with-Os http subprocess:
<verbatim>
V:\dvcs\wc>fossil-ggdb-Os.exe --systemtrace --sqltrace --httptrace http repo in1.txt out1.txt 127.0.0.1 --nossl --localauth
# cgi: GATEWAY_INTERFACE = [CGI/1.0]
# cgi: REQUEST_METHOD = [GET]
# cgi: REQUEST_URI = [/artifact?name=bdbdc5bc614b3003e6e5a275ea8482a9b9059698]
# cgi: PATH_INFO = [/artifact]
# cgi: QUERY_STRING = [name=bdbdc5bc614b3003e6e5a275ea8482a9b9059698]
# cgi: REMOTE_ADDR = [127.0.0.1]
# cgi: HTTP_HOST = [127.0.0.1:8080]
# cgi: name = [bdbdc5bc614b3003e6e5a275ea8482a9b9059698]
SELECT 1 FROM config WHERE name='aux-schema'   AND value<>'2011-04-25 19:50';
SELECT 0 FROM blob WHERE uuid='bdbdc5bc614b3003e6e5a275ea8482a9b9059698';
SELECT rid FROM blob WHERE uuid='bdbdc5bc614b3003e6e5a275ea8482a9b9059698';
SELECT value FROM config WHERE name='localauth';
SELECT uid FROM user WHERE cap LIKE '%s%';
SELECT login FROM user WHERE uid=1;
# login: [Tsul] with capabilities [sx]
SELECT cap FROM user WHERE login = 'nobody';
SELECT cap FROM user WHERE login = 'anonymous';
SELECT uuid FROM blob WHERE rid=2;
SELECT 1 FROM shun WHERE uuid='bdbdc5bc614b3003e6e5a275ea8482a9b9059698';
SELECT value FROM config WHERE name='header';
SELECT value FROM config WHERE name='project-name';
SELECT value FROM config WHERE name='index-page';
SELECT uuid FROM blob WHERE rid=2;
SELECT uuid FROM blob WHERE rid=2;
SELECT filename.name, datetime(event.mtime),
	coalesce(event.ecomment,event.comment),
	coalesce(event.euser,event.user),
	b.uuid  FROM mlink, filename, event, blob a, blob b
	WHERE filename.fnid=mlink.fnid   AND event.objid=mlink.mid   AND a.rid=mlink.fid   AND b.rid=mlink.mid   AND mlink.fid=2;
SELECT value FROM config WHERE name='wiki-use-html';
SELECT substr(tagname, 6, 10000), datetime(event.mtime),
	coalesce(event.euser, event.user)
	FROM tagxref, tag, event WHERE tagxref.rid=2   AND tag.tagid=tagxref.tagid   AND tag.tagname LIKE 'wiki-%'   AND event.objid=tagxref.rid;
SELECT datetime(mtime), user, comment, type, uuid, tagid  FROM event, blob WHERE event.objid=2   AND blob.rid=2;
SELECT target, filename, datetime(mtime), user, src  FROM attachment WHERE src=(SELECT uuid FROM blob WHERE rid=2) ORDER BY mtime DESC /*sort*/;
</verbatim>
And then it's crashed. The input file used was listed above.

Debug output of fossil-without-Os http subprocess differs only in additional lines at the end:
<verbatim>
SELECT srcid FROM delta WHERE rid=2;
SELECT content FROM blob WHERE rid=2 AND size>=0;
SELECT value FROM config WHERE name='footer';
-- LOOKASIDE_USED                 24         95
-- LOOKASIDE_HIT                            728
-- LOOKASIDE_MISS_SIZE                      209
-- LOOKASIDE_MISS_FULL                        0
-- CACHE_USED                  25800
-- SCHEMA_USED                 15608
-- STMT_USED                    3392
-- MEMORY_USED                 94120      99752
-- MALLOC_SIZE                            48000
-- MALLOC_COUNT                  467        479
-- PCACHE_OVFLOW               25384      27568
-- prepared statements            23
</verbatim>

At last, here is the gdb stacktrace of segfault and some other details about fossil-with-Os http subprocess:
<verbatim>
V:\dvcs\wc>gdb fossil-ggdb-Os.exe
GNU gdb (GDB) 7.2
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "mingw32".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from V:\dvcs\wc/fossil-ggdb-Os.exe...done.
(gdb) run http repo in1.txt out1.txt 127.0.0.1 --nossl --localauth
Starting program: V:\dvcs\wc/fossil-ggdb-Os.exe http repo in1.txt out1.txt 127.0.0.1 --nossl --localauth
[New Thread 3792.0xd88]

Program received signal SIGSEGV, Segmentation fault.
0x77c37742 in strcmp () from C:\WINDOWS\system32\msvcrt.dll
(gdb) bt
#0  0x77c37742 in strcmp () from C:\WINDOWS\system32\msvcrt.dll
#1  0x00413a89 in mimetype_from_name (zName=0x6ff440 "\321Д\321\213\320▓\320░") at wbld/doc_.c:302
#2  0x0041e1af in artifact_page () at wbld/info_.c:1250
#3  0x00423081 in process_one_web_page (zNotFound=0x0) at wbld/main_.c:1057
#4  0x00422a6d in main (argc=<value optimized out>, argv=0x3e4088) at wbld/main_.c:290
(gdb) f 1
#1  0x00413a89 in mimetype_from_name (zName=0x6ff440 "\321Д\321\213\320▓\320░") at wbld/doc_.c:302
302           c = fossil_strcmp(zSuffix, aMime[i].zSuffix);
(gdb) l
297         first = 0;
298         last = sizeof(aMime)/sizeof(aMime[0]);
299         while( first<=last ){
300           int c;
301           i = (first+last)/2;
302           c = fossil_strcmp(zSuffix, aMime[i].zSuffix);
303           if( c==0 ) return aMime[i].zMimetype;
304           if( c<0 ){
305             last = i-1;
306           }else{
(gdb) p zSuffix
$1 = "\321Д\321\213\320▓\320░\000■\"\000\330¤\"\000yLC"
(gdb) x/9xb zSuffix
0x22fdac:       0xd1    0x84    0xd1    0x8b    0xd0    0xb2    0xd0    0xb0
0x22fdb4:       0x00
(gdb) p i
$3 = 190
(gdb) p last
$4 = <value optimized out>
(gdb) p aMime[i]
$5 = {zSuffix = 0x38464947 <Address 0x38464947 out of bounds>, size = 1191207223, zMimetype = 0x39384649 <Address 0x39384649 out of bounds>}
(gdb)
</verbatim>
Note that zSuffis = 'фыва' in UTF8, that is the subj. filename.


<hr /><i>anonymous claiming to be tsul added on 2011-05-28 18:26:16 UTC:</i><br />
Fixed in [ea2698e9c7] 2011-05-27 12:16:37

<hr /><i>drh added on 2011-05-28 18:50:35 UTC:</i><br />
OP reports that the latest changes have resolved the problem.

Attachments: