Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
Overview
Comment: | Largely rewrote www/ssl.wiki, it being 7 years since the last update, during which time much has changed in TLS land. Added the initial version of www/tls-nginx.md as a companion article to it, since the topic is too deep to get into within ssl.wiki. Finally, added a new script, tools/fslsrv, referred to by tls-nginx.md, since it's too big to put inline within the article. |
---|---|
Downloads: | Tarball | ZIP archive |
Timelines: | family | ancestors | descendants | both | trunk |
Files: | files | file ages | folders |
SHA3-256: |
062d2bf61bdafe9d9b62e31a48bf19ac |
User & Date: | wyoung 2019-01-20 01:52:36.928 |
Context
2019-01-20
| ||
02:03 | Pointing to evergreen "TLS/HTTPS/SSL Documentation Maintenance" thread from www/tls-nginx.md. ... (check-in: 872d3957 user: wyoung tags: trunk) | |
01:52 | Largely rewrote www/ssl.wiki, it being 7 years since the last update, during which time much has changed in TLS land. Added the initial version of www/tls-nginx.md as a companion article to it, since the topic is too deep to get into within ssl.wiki. Finally, added a new script, tools/fslsrv, referred to by tls-nginx.md, since it's too big to put inline within the article. ... (check-in: 062d2bf6 user: wyoung tags: trunk) | |
2019-01-19
| ||
21:03 | Remove unused variable ... (check-in: 302ce439 user: andygoth tags: trunk) | |
Changes
Added tools/fslsrv.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 | #!/bin/bash BASEPORT=12345 FOSSIL=fossil OLDPID=`pgrep -P 1 fossil` PGARGS="-P 1" if [ "$1" = "-f" ] ; then PGARGS= ; shift ; fi if [ -n "$OLDPID" ] then echo "Killing running Fossil server first..." pkill $PGARGS fossil for i in $(seq 30) do if [ -n "$(pgrep $PGARGS fossil)" ] then if [ $i -eq 1 ] then echo -n "Waiting for it to die..." else echo -n . fi sleep '0.1' else break fi echo done killall -9 fossil 2> /dev/null fi if [ -x ./fossil ] then # We're running from a build tree, so use that version instead FOSSIL=./fossil fi function start_one() { bn=$1 port=$(($BASEPORT + $2)) url="$3" $FOSSIL server --localhost --port $port --scgi \ --baseurl $url --errorlog ~/log/fossil/$bn-errors.log \ ~/museum/$bn.fossil > ~/log/fossil/$bn-stdout.log & echo Fossil server running for $bn, PID $!, port $port. } start_one example 0 https://example.com/code start_one foo 1 https://foo.net |
Changes to www/ssl.wiki.
|
| | | | > > > > > > > > > | > > > > > > > > > > > | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | > > > > > > > > > > > | > > > > > > > > > | > > > > > > | > > > > > > > > > > > > > > > > > > > > > > > > > | > > > > > > > | > > > > > > > > > > > > | > > > | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | > | > > > > > | > > > > > > | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 | <title>TLS and Fossil</title> <h2>Using TLS-Encrypted Communications with Fossil</h2> If you are storing sensitive information in a repository accessible over a network whose integrity you do not fully trust, you should use TLS to encrypt all communications with it. This is most true for repositories accessed over the Internet, especially if they will be accessed from edge networks you do not control, since that admits of various forms of [https://en.wikipedia.org/wiki/Man-in-the-middle_attack|man in the middle attack]. TLS protects the credentials used to access the server, prevents eavesdropping, prevents in-flight data modification, prevents server identify spoofing, and more. There are two major aspects to this, both of which have to be addressed in different ways. Those are the subjects of the next two major sections. <h2 id="client">Client-Side TLS Configuration</h2> Fossil itself has built-in support for TLS on the client side only. That is to say, you can build it against [https://www.openssl.org/|the OpenSSL library], which will allow it to clone and sync with a remote Fossil repository via <tt>https</tt> URIs. The <tt>configure</tt> script will attempt to find OpenSSL on your system automatically, but if necessary, you can specify the location with the <tt>--with-openssl</tt> option. Type <tt>./configure --help</tt> for details. Even if the Fossil build system does manage to find a workable version of OpenSSL, a common situation is that the platform version is outdated in some key way, enough so that you do not want to use it with Fossil. For example, the platform version of OpenSSL might not support any of the [https://en.wikipedia.org/wiki/Cipher_suite|cipher suites] the remote Fossil repository's HTTPS proxy is willing to offer, so that even though both sides are speaking TLS/SSL, they can't come to an agreement on the cryptography. In such cases, you may want to link Fossil to a newer version of OpenSSL than the one provided with your client operating system. You can do this like so: <pre> cd compat tar xf /path/to/openssl-*.tar.gz ln -fs openssl-x.y.z openssl ./config # or, e.g. ./Configure darwin64-x86_64-cc make -j11 cd ../.. ./configure --with-openssl=tree make -j11 </pre> That will get you a Fossil binary statically linked to this in-tree version of OpenSSL. <h3 id="certs">Certificates</h3> To verify the identify of a server, TLS uses [https://en.wikipedia.org/wiki/X.509#Certificates|X.509 certificates]. If you are using a self-signed certificate, you'll be asked if you want to accept the certificate the first time you communicate with the server. Verify the certificate fingerprint is correct, then answer "always" to remember your decision. If you are cloning from or syncing to Fossil servers that use a certificate signed by a [https://en.wikipedia.org/wiki/Certificate_authority|certificate authority] (CA), Fossil needs to know which CAs you trust to sign those certificates. Fossil relies on the OpenSSL library to have some way to check a trusted list of CA signing keys. There are two common ways this fails: # <p>The OpenSSL library Fossil is linked to doesn't have a CA signing key set at all, so that it initially trusts no certificates at all.</p> # <p>The OpenSSL library does have a CA cert set, but your Fossil server's TLS certificate was signed by a CA that isn't in that set.</p> A common reason to fall into the second trap is that you're using certificates signed by a local private CA, as often happens in large enterprises. You can solve this sort of problem by getting your local CA's signing certificate in PEM format and pointing OpenSSL at it: <pre> fossil set --global ssl-ca-location /path/to/local-ca.pem </pre> The use of <tt>--global</tt> with this option is common, since you may have multiple reposotories served under certificates signed by that same CA. A common way to run into the broader first problem is that you're on FreeBSD, which does not install a CA certificate set by default, even as a dependency of the OpenSSL library. If you're using a certificate signed by one of the major public CAs, you can solve this by installing the <tt>ca_root_nss</tt> package. That package contains the Mozilla NSS certificate bundle, which gets installed in a location that OpenSSL checks by default, so you don't need to change any Fossil settings. (This is the same certificate set that ships with Firefox, by the way.) The same sort of thing happens with the Windows build of OpenSSL, but for a different core reason: Windows does ship with a stock CA certificate set, but it's not in a format that OpenSSL understands how to use. Rather than try to find a way to convert the data format, you may find it acceptable to use the same Mozilla NSS cert set. I do not know of a way to easily get this from Mozilla themselves, but I did find a [https://curl.haxx.se/docs/caextract.html|third party source] for the <tt>cacert.pem</tt> file. Install it somewhere on your system, then point Fossil at it like so: <pre> fossil set --global ssl-ca-location /path/to/cacert.pem </pre> Linux platforms tend to provide such a root cert store along with the platform OpenSSL package, either built-in or as a hard dependency. <h4>Client-Side Certificates</h4> You can also use client side certificates to add an extra layer of authentication, over and above Fossil's built in user management. If you are particularly paranoid, you'll want to use this to remove the ability of anyone on the internet from making any request to Fossil. Without presenting a valid client side certificate, the web server won't invoke the Fossil CGI handler. Configure your server to request a client side certificate, and set up a certificate authority to sign your client certificates. For each person who needs to access the repository, create a private key and certificate signed with that CA. The PEM encoded private key and certificate should be stored in a single file, simply by concatenating the key and certificate files. Specify the location of this file with the <tt>ssl-identity</tt> setting, or the <tt>--ssl-identity</tt> option to the <tt>clone</tt> command. If you've password protected the private key, the password will be requested every time you connect to the server. This password is not stored by fossil, as doing so would defeat the purpose of having a password. If you attempt to connect to a server which requests a client certificate, but don't provide one, fossil will show an error message which explains what to do to authenticate with the server. <h2 id="server">Server-Side TLS Configuration</h2> Fossil's built-in HTTP server feature does not currently have a built-in way to serve via HTTP over TLS, a.k.a. HTTPS, even when you've linked Fossil to OpenSSL. To serve a Fossil repository via HTTPS, you must put it behind some kind of HTTPS proxy. <h3 id="stunnel">stunnel Alone</h3> Conceptually, the simplest option is to [https://www.stunnel.org/|set up stunnel]. A typical configuration is to run Fossil as an HTTP server bound to localhost only, then export access to it via HTTPS with stunnel encrypting access to Fossil instance hiding behind it. The difficulty comes in configuring it, which really wants a guide that no one has written for us yet. Until that appears, you'll have to read the stunnel documentation and that of your TLS certificate provider to work out how to set this up. <h3 id="althttpd">stunnel + althttpd</h3> The public SQLite and Fossil web sites can't just use stunnel + Fossil because parts of the web site are static, served by [https://www.sqlite.org/docsrc/file/misc/althttpd.c|a separate web server called <tt>althttpd</tt>], written by the primary author of both SQLite and Fossil. It is a lightweight HTTP-only web server. It handles the static HTTP hits on <tt>sqlite.org</tt> and <tt>fossil-scm.org</tt>, delegating HTTPS and dynamic Fossil content hits to stunnel and Fossil. The only documentation for althttpd currently is in its header comment. As is typical for drh software, althttpd is a single-file C program, so that at worst, you just have to read its code to understand it. <h3 id="nginx">nginx</h3> If your needs are more complex than althttpd can handle or you'd prefer to use only software available in your server operating system's package repository, we recommend that you step up to [http://nginx.org/|nginx]. Setting this up is complex enough that we cover it [./tls-nginx.md|in a separate document]. <h2 id="enforcing">Enforcing TLS Access</h2> To use TLS encryption in cloning and syncing to a remote Fossil repository, be sure to use the <tt>https:</tt> URI scheme in <tt>clone</tt> and <tt>sync</tt> commands. If your server is configured to serve the repository via both HTTP and HTTPS, it's easy to accidentally use unencrypted HTTP if you forget the all-important 's'. There is a setting in the Fossil UI under Admin → Access called "Redirect to HTTPS on the Login page." This setting is not enabled by default. This setting does not automatically upgrade clones and syncs done via the <tt>http</tt> URI scheme. It only affects web UI access to the Fossil repository. <b id="rloop">WARNING:</b> Never enable this setting when running Fossil behind an HTTPS proxy with Fossil running underneath it via HTTP or SCGI. This will cause an infinite redirect loop any time someone tries to log into the web UI. Fossil sees that it's being accessed via HTTP, so it redirects the browser to an HTTPS equivalent URL, which causes the client to hit the HTTPS front end proxy up again for access, which causes Fossil to see that it's being accessed via HTTP, so it redirects the client to...'round and 'round it goes until the web browser detects it's in a redirect loop and gives up. If you wish to enforce TLS-only access to a Fossil web server, it is best done at the HTTPS front-end proxy layer, not by use of Fossil-level settings like this one. The [./tls-nginx.md|nginx TLS proxy guide] shows one way to achieve this, for example. <h2>Terminology Note</h2> This document is called <tt>ssl.wiki</tt> for historical reasons. The TLS protocol was originally called SSL, and it went through several revisions before being replaced by TLS. Years before this writing, SSL finally became entirely obsolete due to weaknesses in the protocol fixed in the later TLS series of protocols. Some people still use the term "SSL" when they actually mean "TLS," but in the Fossil project, we always use "TLS" except when we must preserve some sort of historical compatibility, as with this document's name in order to avoid broken external URLs. The Fossil TLS-related settings also often use "`ssl`" in their names, for the same reason. This series of protocols is also called "HTTPS" after the URI scheme used to specify "HTTP over TLS." |
Added www/tls-nginx.md.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 | # Proxying Fossil via HTTPS with nginx One of the [many ways](./ssl.wiki) to provide TLS-encrypted HTTP access (a.k.a. HTTPS) to Fossil is to run it behind a web proxy that supports TLS. This document explains how to use the powerful [nginx web server](http://nginx.org/) to do that. ## Benefits This scheme is complicated, even with the benefit of this guide and pre-built binary packages. Why should you put up with this complexity? Because it gives many benefits that are difficult or impossible to get with the less complicated options: * **Power** — nginx is one of the most powerful web servers in the world. The chance that you will run into a web serving wall that you can’t scale with nginx is very low. To give you some idea of the sort of thing you can readily accomplish with nginx, your author runs a single public web server that provides transparent name-based virtual hosting for four separate domains: * One is entirely static, not involving any dynamic content or Fossil integration at all. * Another is served almost entirely by Fossil, with a few select static content exceptions punched past Fossil, which are handled entirely via nginx. * The other two domains are aliases for one another — e.g. `example.com` and `example.net` — with most of the content being static. This pair of domains has three different Fossil repo proxies attached to various sections of the URI hierarchy. All of this is done with minimal configuration repetition between the site configurations. * **Integration** — Because nginx is so popular, it integrates with many different technologies, and many other systems integrate with it in turn. This makes it great middleware, sitting between the outer web world and interior site services like Fossil. It allows Fossil to participate seamlessly as part of a larger web stack. * **Availability** — nginx is already in most operating system binary package repositories, so you don’t need to go out of your way to get it. ## Fossil Remote Access Methods Fossil provides four major ways to access a repository it’s serving remotely, three of which you can use with nginx: * **HTTP** — Fossil has a built-in HTTP server: `fossil server`. While this method is efficient and it’s possible to use nginx to proxy access to another HTTP server, this option is overkill for our purposes. nginx is itself a fully featured HTTP server, so we will choose in this guide not to make nginx reinterpret Fossil’s implementation of HTTP. * **CGI** — This method is simple but inefficient, because it launches a separate Fossil instance on every HTTP hit. Since Fossil is a relatively small self-contained program, and it’s designed to start up quickly, this method can work well in a surprisingly large number of cases. Nevertheless, we will avoid this option in this document because we’re already buying into a certain amount of complexity here in order to gain power. There’s no sense in throwing away any of that hard-won performance on CGI overhead. * **SCGI** — The [SCGI protocol][scgi] provides the simplicity of CGI without its performance problems. * **SSH** — This method exists primarily to avoid the need for HTTPS in the first place. There is probably a way to get nginx to proxy Fossil to HTTPS via SSH, but it would be pointlessly complicated. SCGI it is, then. # Installing The first step is to install the pieces we’ll be working with. This varies on different operating systems, so to avoid overcomplicating this guide, we’re going to assume you’re using Ubuntu Server 18.04 LTS, a common Tier 1 offering for [virtual private servers][vps]. SSH into your server, then say: $ sudo apt install certbot fossil nginx For other operating systems, simply visit [the front Certbot web page][cb] and tell it what OS and web stack you’re using. Chances are good that they’ve got a good guide for you already. # Running Fossil in SCGI Mode You presumably already have a working Fossil configuration on the public server you’re trying to set up and are just following this guide to replace HTTP service with HTTPS. (You can adjust the advice in this guide to get both HTTP *and* HTTPS service on the same site, but I strongly recommend that you do not do that: the good excuses remaining for continuing to allow HTTP on public web servers are running thin these days.) I run my Fossil SCGI server instances with a variant of [the `fslsrv` shell script](/file/tools/fslsrv) currently hosted in the Fossil source code repository. You’ll want to download that and make a copy of it, so you can customize it to your particular needs. This script allows running multiple Fossil SCGI servers, one per repository, each bound to a different high-numbered `localhost` port, so that only nginx can see and proxy them out to the public. The “`example`” repo is on TCP port localhost:12345, and the “`foo`” repo is on localhost:12346. As written, the `fslsrv` script expects repositories to be stored in the calling user’s home directory under `~/museum`, because where else do you keep Fossils? That home directory also needs to have a directory to hold log files, `~/log/fossil/*.log`. Fossil doesn’t put out much logging, but when it does, it’s better to have it captured than to need to re-create the problem after the fact. The use of `--baseurl` in this script lets us have each Fossil repository mounted in a different location in the URL scheme. Here, for example, we’re saying that the “`example`” repository is hosted under the `/code` URI on its domains, but that the “`foo`” repo is hosted at the top level of its domain. You’ll want to do something like the former for a Fossil repo that’s just one piece of a larger site, but the latter for a repo that is basically the whole point of the site. This script’s automatic restart feature makes Fossil upgrades easy: $ cd ~/src/fossil/trunk ; fossil up ; make ; killall fossil ; sudo make install ; fslsrv I’ve written that as a single long command because I keep it in the history for my Fossil servers, so I can just run it again from history. You could put it in a shell script instead. The `killall fossil` step is needed only on OSes that refuse to let you replace a running binary on disk. As written, the `fslsrv` script assumes a Linux environment. It expects `/bin/bash` to exist, and it depends on non-POSIX tools like `pgrep`. It shouldn’t be difficult to port to very different systems, like macOS or the BSDs. # Configuring Let’s Encrypt, the Easy Way If your web serving needs are simple, [Certbot][cb] can configure nginx for you and keep its certificates up to date. The details are pretty much as in the Certbot documentation for [nginx on Ubuntu 18.04 LTS guide][cbnu], except that where they recommend that you use the first-party Certbot packages, we’ve found that the ones that come with Ubuntu work just fine. The primary local configuration you need is to tell nginx how to proxy certain URLs down to the Fossil instance you started above with the `fslsrv` script: location / { include scgi_params; scgi_pass 127.0.0.1:12345; scgi_param HTTPS "on"; scgi_param SCRIPT_NAME ""; } The TCP port number in that snippet is the key: it has to match the port number generated by `fslsrv` from the base port number passed to the `start_one` function. # Configuring Let’s Encrypt, the Hard Way If you’re finding that you can’t get certificates to be issued or renewed using the Easy Way instructions, the problem is usually that your nginx configuration is too complicated for Certbot’s `--nginx` plugin to understand. It attempts to rewrite your nginx configuration files on the fly to achieve the renewal, and if it doesn’t put its directives in the right locations, the ACME verification steps can fail. Your author’s configuration, glossed above, is complicated enough that the current version of Certbot (0.28 at the time of this writing) can’t cope with it. That’s the primary motivation for me to write this guide: I’m addressing the “me” years hence who needs to upgrade to Ubuntu 20.04 or 22.04 LTS and has forgotten all of this stuff. 😉 ## Step 1: Shifting into Manual The first thing to do is to turn off all of the Certbot automation, because it’ll only get in our way. First, disable the Certbot package’s automatic background updater: $ sudo systemctl disable certbot.timer Next, edit `/etc/letsencrypt/renewal/example.com.conf` to disable the nginx plugins. You’re looking for two lines setting the “install” and “auth” plugins to “nginx”. You can comment them out or remove them entirely. ## Step 2: Configuring nginx On Ubuntu systems, at least, the primary user-level configuration file is `/etc/nginx/sites-enabled/default`. For a configuration like I described at the top of this article, I recommend that this file contain only a list of include statements, one for each site that server hosts: include local/example include local/foo Those files then each define one domain’s configuration. Here, `/etc/nginx/local/example` contains the configuration for `*.example.com` and `*.example.net`; and `local/foo` contains the configuration for `*.foo.net`. Here’s an example configuration: server { server_name .foo.net; include local/tls-common; charset utf-8; access_log /var/log/nginx/foo.net-https-access.log; error_log /var/log/nginx/foo.net-https-error.log; # Bypass Fossil for the static Doxygen docs location /doc/html { root /var/www/foo.net; location ~* \.(html|ico|css|js|gif|jpg|png)$ { expires 7d; add_header Vary Accept-Encoding; access_log off; } } # Redirect everything else to the Fossil instance location / { include scgi_params; scgi_pass 127.0.0.1:12345; scgi_param HTTPS "on"; scgi_param SCRIPT_NAME ""; } } server { server_name .foo.net; root /var/www/foo.net; include local/http-certbot-only; access_log /var/log/nginx/foo.net-http-access.log; error_log /var/log/nginx/foo.net-http-error.log; } Notice that we need two `server { }` blocks: one for HTTPS service, and one for HTTP-only service: ### HTTP over TLS (HTTPS) Service The first `server { }` block includes this file, `local/tls-common`: listen 443 ssl; ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem; ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem; ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; ssl_protocols TLSv1 TLSv1.1 TLSv1.2; ssl_ciphers "ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SH ssl_session_cache shared:le_nginx_SSL:1m; ssl_prefer_server_ciphers on; ssl_session_timeout 1440m; These are the common TLS configuration parameters used by all domains hosted by this server. Since all of those domains share a single TLS certificate, we reference the same `example.com/*.pem` files written out by Certbot here. We also reference the common server-specific Diffie-Hellman parameter file written by the Let’s Encrypt package when it’s initially installed. The `ssl_protocols` and `ssl_ciphers` lines are prone to bit-rot: as new attacks on TLS and its associated technologies are discovered, this configuration is likely to need to change. Even if we fully succeed in [keeping this document up-to-date](#evolution), the nature of this guide is to recommend static configurations for your server. You will have to keep an eye on this sort of thing and evolve your local configuration as the world changes around it. Running a TLS certificate checker against your site occasionally is a good idea. The most thorough service I’m aware of is the [Qualys SSL Labs Test][qslt], which gives the site I’m basing this guide on an “A” rating at the time of this writing. I assume you’re taking my advice to serve only the least amount of stuff over HTTP that you can get away with. Certbot’s ACME HTTP-01 authentication scheme is one of those few things. ### HTTP-Only Service While we’d prefer not to offer HTTP service at all, we need to do so for two reasons, one temporary and the other going forward indefinitely. First, until we get Let’s Encrypt certificates minted and configured properly, we can’t use HTTPS yet at all. Second, the Certbot ACME HTTP-01 challenge used by the Let’s Encrypt service only runs over HTTP, because it has to work before HTTPS is working, or after a certificate is accidentally allowed to lapse. This is the protocol Let’s Encrypt uses to determine whether we actually have control over the domains we want our certificate to be minted for. Let’s Encrypt will not just let you mint certificates for `google.com` and `paypal.com`! So, from the second `service { }` block, we include this file to set up the minimal HTTP service we reqiure, `local/http-certbot-only`: listen 80; listen [::]:80; # This is expressed as a rewrite rule instead of an "if" because # http://wiki.nginx.org/IfIsEvil #rewrite ^(/.well-known/acme-challenge/.*) $1 break; # Force everything else to HTTPS with a permanent redirect. #return 301 https://$host$request_uri; As written above, this configuration does nothing other than to tell nginx that it’s allowed to serve content via HTTP on port 80 as well. We’ll uncomment the `rewrite` and `return` directives below, when we’re ready to begin testing. #### Why the Repitition? You need to do much the same sort of thing as above for each domain name hosted by your nginx server. You might being to wonder, then, why I haven’t factored some of those directives into the included files `local/tls-common` and `local/http-certbot-only`. For example, why can’t the second HTTP-only `server { }` block above just be these two lines: server_name .foo.net; include local/http-certbot-only; Then in `local/http-certbot-only`, we’d like to say: root /var/www/$host; access_log /var/log/nginx/$host-http-access.log; error_log /var/log/nginx/$host-http-error.log; Sadly, nginx doesn’t allow variable subtitution into any of these directives. As I understand it, allowing that would make nginx slower, so we must largely repeat these directives in each HTTP `server { }` block. These configurations are, as shown, as small as I know how to get them. If you know of a way to reduce some of this repitition, [I solicit your advice][fd]. ## Step 3: Dry Run We want to first request a dry run, because Let’s Encrypt puts some rather low limits on how often you’re allowed to request an actual certificate. You want to be sure everything’s working before you do that. You’ll run a command something like this: $ sudo certbot certonly --webroot --dry-run \ --webroot-path /var/www/example.com \ -d example.com -d www.example.com \ -d example.net -d www.example.net \ --webroot-path /var/www/foo.net \ -d foo.net -d www.foo.net There are two key options here. First, we’re telling Certbot to use its `--webroot` plugin instead of the automated `--nginx` plugin. With this plugin, Certbot writes the ACME HTTP-01 challenge files to the static web document root directory behind each domain. For this example, we’ve got two web roots, one of which holds documents for two different second-level domains (`example.com` and `example.net`) with `www` at the third level being optional. This is a common sort of configuration these days, but you needn’t feel that you must slavishly imitate it; the other web root is for an entirely different domain, also with `www` being optional. Since all of these domains are served by a single nginx instance, we need to give all of this in a single command, because we want to mint a single certificate that authenticates all of these domains. The second key option is `--dry-run`, which tells Certbot not to do anything permanent. We’re just seeing if everything works as expected, at this point. ### Troubleshooting the Dry Run If that didn’t work, try creating a manual test: $ mkdir -p /var/www/example.com/.well-known/acme-challenge $ echo hi > /var/www/example.com/.well-known/acme-challenge/test Then try to pull that file over HTTP — not HTTPS! — as `http://example.com/.well-known/acme-challenge/test`. I’ve found that using Firefox and Safari is better for this sort of thing than Chrome, because Chrome is more aggressive about automatically forwarding URLs to HTTPS even if you requested “`http`”. In extremis, you can do the test manually: $ telnet foo.net 80 GET /.well-known/acme-challenge/test HTTP/1.1 Host: example.com HTTP/1.1 200 OK Server: nginx/1.14.0 (Ubuntu) Date: Sat, 19 Jan 2019 19:43:58 GMT Content-Type: application/octet-stream Content-Length: 3 Last-Modified: Sat, 19 Jan 2019 18:21:54 GMT Connection: keep-alive ETag: "5c436ac2-4" Accept-Ranges: bytes hi You’re looking for that “hi” line at the end and the “200 OK” response here. If you get a 404 or other error response, you need to look into your web server logs to find out what’s going wrong. Note that it’s important to do this test with HTTP/1.1 when debugging a name-based virtual hosting configuration like this, if the test domain is one of the secondary names, as in the example above, `foo.net`. If you’re still running into trouble, the log file written by Certbot can be helpful. It tells you where it’s writing it early in each run. ## Step 4: Getting Your First Certificate Once the dry run is working, you can drop the `--dry-run` option and re-run the long command above. (The one with all the `--webroot*` flags.) This should now succeed, and it will save all of those flag values to your Let’s Encrypt configuration file, so you don’t need to keep giving them. ## Step 5: Test It Edit the `local/http-certbot-only` file and uncomment the `redirect` and `return` directives, then restart your nginx server and make sure it now forces everything to HTTPS like it should: $ sudo systemctl restart nginx Test ideas: * Visit both Fossil and non-Fossil URLs * Log into the repo, log out, and log back in * Clone via `http`: ensure that it redirects to `https`, and that subsequent `fossil sync` commands go directly to `https` due to the 301 permanent redirect. This forced redirect is why we don’t need the Fossil Admin → Access "Redirect to HTTPS on the Login page" setting to be enabled. Not only is it unnecessary with this HTTPS redirect at the front-end proxy level, it would actually [cause an infinite redirect loop if enabled](./ssl.wiki#rloop). ## Step 6: Renewing Automatically Now that the configuration is solid, you can renew the LE cert and restart nginx with two short commands, which are easily automated: sudo certbot certonly --webroot sudo systemctl restart nginx I put those in a script in the `PATH`, then arrange to call that periodically. Let’s Encrypt doesn’t let you renew the certificate very often unless forced, and when forced there’s a maximum renewal counter. Nevertheless, some people recommend running this daily and just letting it fail until the server lets you renew. Others arrange to run it no more often than it’s known to work without complaint. Suit yourself. ----------- <a id=”evolution”></a> **Document Evolution** TLS and web proxying are a constantly evolving technology. This article replaces my [earlier effort][2016], which had whole sections that were basically obsolete within about a year of posting it. Two years on, and I was encouraging readers to ignore about half of that HOWTO. I am now writing this document about 3 years later because Let’s Encrypt deprecated key technology that HOWTO depended on, to the point that following that old HOWTO is more likely to confuse than enlighten. There is no particularly good reason to expect that this sort of thing will not continue to happen, so this effort is expected to be a living document. If you do not have commit access on the `fossil-scm.org` repository to update this document as the world changes around it, you can discuss this document [on the forum][fd]. This document’s author keeps an eye on the forum and expects to keep this document updated with ideas that appear in that thread. [2016]: https://www.mail-archive.com/fossil-users@lists.fossil-scm.org/msg22907.html [cb]: https://certbot.eff.org/ [cbnu]: https://certbot.eff.org/lets-encrypt/ubuntubionic-nginx [fd]: https://fossil-scm.org/forum/forumpost/XXXXXXXX [qslt]: https://www.ssllabs.com/ssltest/ [scgi]: https://en.wikipedia.org/wiki/Simple_Common_Gateway_Interface [vps]: https://en.wikipedia.org/wiki/Virtual_private_server |