Fossil

Check-in [4bbd5c3e]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Fixed bug in test files generated in commit-warning tests that caused some UCS-2 (aka UTF-16) files to have an odd byte length and be seen as binary instead of Unicode.
Downloads: Tarball | ZIP archive | SQL archive
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA3-256:4bbd5c3e47544f5f06ed52475ee2b4cd49eb24fe23f35f6a50087b85ea2fe22f
User & Date: rberteig 2017-03-31 00:11:45
Context
2017-04-03
11:40
Add support for GIT comment tags, as suggested here. Implementation by Roy Marples, but modified to use the (existing) 'T' card in stead of the 'C' card. check-in: 3bba2a6e user: jan.nijtmans tags: trunk
2017-03-31
00:11
Fixed bug in test files generated in commit-warning tests that caused some UCS-2 (aka UTF-16) files to have an odd byte length and be seen as binary instead of Unicode. check-in: 4bbd5c3e user: rberteig tags: trunk
2017-03-30
18:58
Cherry-pick Solaris build fix from SQLite trunk http://www.sqlite.org/src/info/c8000e94cca59dab check-in: 54cf4397 user: jan.nijtmans tags: trunk
Changes
Hide Diffs Unified Diffs Ignore Whitespace Patch

Changes to test/commit-warning.test.

85
86
87
88
89
90
91
92
93


94
95
96
97
98
99
100
101
102
103
104
105
106
...
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
protOut $emoji
write_file utf-8-emoji $emoji
write_file utf-8-bom-emoji "\xef\xbb\xbf$emoji"

# UTF-16 uses 16-bit values to cover all valid unicode code points
# from U+0 to U+10FFFF, using surrogate pairs to escape the BMP.
# Interchange require knowing (and preserving) byte order.
write_file utf-16le-hello "h\x00e\x00l\x00l\x00o\x00\n\x00"
write_file utf-16be-hello "\x00h\x00e\x00l\x00l\x00o\x00\n"


set bomLE "\xff\xfeh"
set bomBE "\xfe\xffh"
write_file utf-16le-bomle-hello "$bomLE\x00e\x00l\x00l\x00o\x00\n\x00"
write_file utf-16be-bombe-hello "$bomBE\x00h\x00e\x00l\x00l\x00o\x00\n"
write_file utf-16le-bombe-hello "$bomBE\x00e\x00l\x00l\x00o\x00\n\x00"
write_file utf-16be-bomle-hello "$bomLE\x00h\x00e\x00l\x00l\x00o\x00\n"
set le16 [read_file [file join $testdir utf16le.txt]]
set be16 [read_file [file join $testdir utf16be.txt]]
write_file utf-16le.txt $le16 
write_file utf-16be.txt $be16
write_file utf-nobom-16le.txt [string range $le16 2 end] 
write_file utf-nobom-16be.txt [string range $be16 2 end]
#write_file [file join $::env(TEMP) utf-nobom-16le.txt] [string range $le16 2 end] 
................................................................................
0\tempty\t
0\tline-0064\t
0\tline-1024\t
0\tline-4096\t
1\tline-64K\tlong lines
1\tline-8192\tlong lines
0\tplain.txt\t
1\tutf-16be-bombe-hello\tbinary data
1\tutf-16be-bomle-hello\tbinary data
1\tutf-16be-hello\tbinary data
1\tutf-16be.txt\tUnicode
1\tutf-16le-bombe-hello\tUnicode
1\tutf-16le-bomle-hello\tUnicode
1\tutf-16le-hello\tbinary data
1\tutf-16le.txt\tUnicode
0\tutf-8-bom-emoji\t







|
|
>
>
|
|
|
|
|
|







 







|
|







85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
...
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
protOut $emoji
write_file utf-8-emoji $emoji
write_file utf-8-bom-emoji "\xef\xbb\xbf$emoji"

# UTF-16 uses 16-bit values to cover all valid unicode code points
# from U+0 to U+10FFFF, using surrogate pairs to escape the BMP.
# Interchange require knowing (and preserving) byte order.
set hello16LE "h\x00e\x00l\x00l\x00o\x00\n\x00"
set hello16BE "\x00h\x00e\x00l\x00l\x00o\x00\n"
write_file utf-16le-hello $hello16LE
write_file utf-16be-hello $hello16BE
set bomLE "\xff\xfe"
set bomBE "\xfe\xff"
write_file utf-16le-bomle-hello "$bomLE$hello16LE"
write_file utf-16be-bombe-hello "$bomBE$hello16BE"
write_file utf-16le-bombe-hello "$bomBE$hello16LE"
write_file utf-16be-bomle-hello "$bomLE$hello16BE"
set le16 [read_file [file join $testdir utf16le.txt]]
set be16 [read_file [file join $testdir utf16be.txt]]
write_file utf-16le.txt $le16 
write_file utf-16be.txt $be16
write_file utf-nobom-16le.txt [string range $le16 2 end] 
write_file utf-nobom-16be.txt [string range $be16 2 end]
#write_file [file join $::env(TEMP) utf-nobom-16le.txt] [string range $le16 2 end] 
................................................................................
0\tempty\t
0\tline-0064\t
0\tline-1024\t
0\tline-4096\t
1\tline-64K\tlong lines
1\tline-8192\tlong lines
0\tplain.txt\t
1\tutf-16be-bombe-hello\tUnicode
1\tutf-16be-bomle-hello\tUnicode
1\tutf-16be-hello\tbinary data
1\tutf-16be.txt\tUnicode
1\tutf-16le-bombe-hello\tUnicode
1\tutf-16le-bomle-hello\tUnicode
1\tutf-16le-hello\tbinary data
1\tutf-16le.txt\tUnicode
0\tutf-8-bom-emoji\t