Hello,

I have a fossil repository served by one https web server with a self-signed certificate.

I want to script a clone on windows but I have problems with automatically responding to the question: "accept the certificate?"

Some questions:

1. How to script (in tcl) the acceptance to the question?

2. If I have the certificate private key, how exactly do I check that one certificate corresponds with the other?

3. In my computer, I had already in the past accepted the certificate and it does not ask for it again, so I cannot test. How do I remove the certificate acceptance?

1. How to script (in tcl) the acceptance to the question?

Something similar to how it'd be done on the command-line 'echo "yes" | fossil ...'. Basically, supplying input to the anticipated prompt.

1. How do I remove the certificate acceptance?

If I recall it correctly, it's stored in per-user .fossil config db file (not the repo or checkout db). I don't think there's a UI to clear the accepted cert directly. But it's a Sqlite db, in my case I just deleted the row corresp. to the host using sqlite3 client.

P.S. This Markdown parser is broken. Using pipe char in inline code (backtick'ed) still renders the paragraph in two columns (as a table header?).

P.S. This Markdown parser is broken. Using pipe char in inline code (backtick'ed) still renders the paragraph in two columns (as a table header?).

Not all the time ;). It's unfortunately somewhat unpredictable. Someone has created a patch for that but their contributor agreement is still on the way, so that patch has not yet been integrated.

Please remind me of where to find this patch, who created it, and when they sent in their contributor agreement.

Thank you both for pursuing this. I could use a fix for this issue in documentation for a C compiler, where I’ve been using ¦ hackery to document its bitwise and logical OR operators, to avoid this parsing bug.

Y'all please experiment with the latest trunk and make sure my changes based on aitap's patch don't break anything. I observe that it does fix the formatting in the penultimate paragraph of post e39020f82d7d510e on the SQLite forum.

Y'all please experiment with the latest trunk and make sure my changes based on aitap's patch don't break anything.

It also fixes /forumpost/8055955a18 :-D.

Same with inline code between backticks 'this | that'.

Escaped pipe char is supposed to render as pipe by definition 'this | that'.

I'd expect a table markdown to begin with a pipe char in the first position on the line and also between column text. And render as html table with th and td cells.

Not that I'd try to really type that way in a webform, but maybe pasting it from elsewhere (SQLite output?). I guess, table markdown is an extension, it's a question of how Fossil implements it.

    When I type 'this | that', I expect to see an unescaped pipe char....

Do other Markdown engines work this way? What does GitHub do?

I tested the exact same post (12) [source] on GitHub.

The rendered text in GitHub's Issue Preview is as follows (code-fenced here for verbatim view):

When I type 'this | that', I expect to see an unescaped pipe char between "this" and "that".

All other paragraphs are rendered similarly to Fossil. The table markdown seems to be signaled by ----|, that is the underline of the header row. The first column is allowed to begin without a leading pipe char, and can have leading whitespace.

Here's markdown for a single-column table (see [source]), as recognized by GitHub's markdown parser:

Should we remove the ----|, it then this markdown renders there as normal body text:

When I type 'this

that', I expect to see an unescaped pipe char between "this" and "that".

You have used apostrophes there, not backticks. Apostrophes don't get special treatment with regards to escaping of their content.

That's as may be. To me the key question is, why does Fossil's Markdown processor try to interpret a random pipe char in the middle of a line as part of a table, escaping or no? Surely the table logic should only come in when there is a pipe either at the start of the line or following some number of whitespace characters?

Alas, the CommonMark people are still arguing about tables, not yet having standardized the feature, so we can't lean on them for a ruling.

FYI, below is a minimal way to convert a SQLite output table (.header on; .separator |; select * from event limit 2; ) into nicely aligned GitHub's markdown table:

Basically, at the very least adding '-|' for each column of the header (looks like the trailing pipe char for cols > 1 is also optional).

Thank you! I've verified the fix.

How do I remove the certificate acceptance?

View the exceptions by running:

    fossil tls-config show

Remove an exception by running:

    fossil tls-config remove-exception $DOMAIN

with a self-signed certificate.

In a world with Let’s Encrypt, the only remaining use I can see for self-signed certs is inside the firewall, and then only in orgs too small to be able to justify running an internal CA, trusted internally by pushing its public signing key out to all the site browsers.

If that’s your case, fine, but otherwise, I’d strongly suggest getting off self-signed certs. There’s a good reason why OpenSSL warns about them, and why Fossil passes on that warning.

If I have the certificate private key...

...then you’re breaking the TLS security model if a copy ever gets to the client side, at which point one wonders why you’re bothering with TLS in the first place.

It’s called the private key for a reason; keep it private!

What you want here instead is to copy the server’s public key fingerprint to the client and verify that.

Your next question is “How?” of course. I expect the answer requires a new feature in fossil, something like:

   C:\> fossil clone --server-fingerprint HEXBARF https://...

a new feature in fossil...

It looks like you need to chain a bunch of OpenSSL calls to achieve this:

1. We already call SSL_get_certificate() at the beginning of a TLS conversation with the remote Fossil server, yielding an X509 structure.

2. We can then pass that to X509_get_X509_PUBKEY() to get an X509_PUBKEY structure.

3. We must then call i2d_X509_PUBKEY() on that to get the internal public key representation converted to DER format. (I believe "i2d" means "internal to DER".)

4. An X.509 certificate fingerprint is a hash of the DER-format version of that cert. Therefore, we must hash the prior value using whatever algorithm was used to produce the fingerprint in the first place. Originally, this was MD5, but SHA-1 and SHA-256 are more common these days. We can infer which one is used by the length of the passed fingerprint.

5. Compare that hash to the value presented on the command line. If it doesn't match, either the key must be forged, or the user passed the wrong fingerprint.

Easy! 😜

More thoughts:

1. Implicit in the above is that if the fingerprint matches, we should skip the warning about self-signed certs. The user has proven that they know the certificate is acceptable with astronomical odds against a false positive. This solves the OP's problem, allowing their Tcl script to pass a parameter that safely suppresses the warning.

2. I'm torn on whether the feature should store the passed fingerprint in the repo clone's config area or to store the implicitly-accepted certificate in the list of accepted certs as if the user had explicitly accepted it:

   • As designed above, the fingerprint is passed as part of the cloning operation, so doesn't that make the setting per-repo?
   • If we aren't going to store the implicitly accepted cert globally, then we need to keep the fingerprint for later checks, to prevent Fossil from griping about it being untrusted in the "Tell the user about the failure and ask what to do" case within ssl_open(), in http_ssl.c.
   • Dare we store the certificate globally when it's implicitly accepted like this? If I clone another repo on the same server but I don't pass the --server-fingerprint option to clone, should it implicitly accept that same cert for the new repo as well, or do we expect it to warn of a self-signed cert in that case?

3. Do we put this fingerprint check before the call to SSL_get_verify_result(), making it a simply bypass for the certificate check, or do we put it inside the failure condition, perhaps as an "else if" condition after the call to ssl_certificate_exception_exists()?

In a world with Let’s Encrypt, the only remaining use I can see for self-signed certs is inside the firewall, and then only in orgs too small to be able to justify running an internal CA, trusted internally by pushing its public signing key out to all the site browsers.

There is one more perfectly valid use for self-signed certs - distrusting public certificate authorities. Especially, that without a special configuration openssl simply trusts all CAs that are defined in the system - it is impossible to specify, with which CA the server cert should be signed by. And there were already cases of exploiting this fact - a malicious CAs signed certs for some agencies even for domains like google.com. Specifying the only trusted CA solves all this problems (and it makes using self-signed certs really easy).

distrusting public certificate authorities.

Thus my point about "running an internal CA." If that is the only CA you trust, then that's your call.

without a special configuration openssl simply trusts all CAs that are defined in the system

OpenSSL doesn't ship with any pre-trusted CAs. (Thus this near-FAQ.) It is likely your OS packager that provided that list, so your beef is with them, not with OpenSSL or with the CA trust system.

And you aren't required to accept that list. The simplest way of editing it appears to be:

   $ openssl x509 -addreject cert-i-distrust.pem /path/to/cacert.pem

If you want the rejected certs fully out of the file, I think you have to deconstruct it to individual certs and then recompile it into a new PEM file.

Then, obviously, you need to mark the package owning that file as non-upgradeable, since you've taken over manual maintenance of its contents.

Probably simpler to just run your own CA after all. :)

I did not mean distrusting all public CAs globally for any app in the system. Eg. I don't care what cert the news site is using (I don't even care if they use https at all).

But I think it would be valuable to be able to point out a CA that I trust (or at least that I know is the one that signed my server's cert) when connecting to possibly critical resource like fossil repo. And I don't need to run my own CA for that...

it would be valuable to be able to point out a CA that I trust (or at least that I know is the one that signed my server's cert) when connecting to possibly critical resource like fossil repo

Fingerprints are useful for that case as well.

Even with a public CA-signed cert, you could use this proposed fingerprinting feature: you are saying, "Only accept the certificate if it has this fingerprint."

That means if you're using an often-rotating certificate (e.g. Let's Encrypt) then you have to update your clone commands every time it changes, but security is inherently inconvenient. If there are no barriers, there is no security.

you have to update your clone commands

Hmmmm...we'll also need an equivalent of the remote-url command, so I'm thinking this flag should be called --remote-fingerprint to fit existing conventions. Then we add fossil remote-fingerprint NEW_HEXBARF for use within a checkout of an existing clone to update the fingerprint stored in the repo file each time it changes.

It should all be sub-commands under the existing "fossil tls-config" command. And we already have the "remove-fingerprint" command there, except that it is currently called "remove-exception" because the records are only consulted when the cert cannot be validated using CAs.

Once the --server-fingerprint is implemented, the message given by

 fossil clone

should be:

 Unable to verify SSL cert from ....
 use 'fossil clone --server-fingerprint HEXBARF'

where `HEXBARF' should be the real fingerprint to use.

In this way, it is not necessary to use strange commands like:

 openssl x509 -in CERT.pem -noout -sha256 -fingerprint

If you need total security, just do this operation once from a local computer or secure channel and write down the number.

While code-diving to look into this, I discovered a forgotten command-line option: --accept-any-cert. If you add the --accept-any-cert command-line option to your "fossil clone" command, it will accept whatever self-signed cert the server sends back.

Maybe we should document the --accept-any-cert command-line option someplace where people can easily find it?

That may solve the OP's immediate problem, but I think we still need a feature that says "Automatically accept the server's public cert if it looks like this..."

How does the user get the SHA3-256 hash of the cert to put on the command line?

Maybe a new command to add an exception for a given domain? Something like this:

    fossil tls-config add-exception https://selfsigned.new/

This would prompt Fossil to open an SSL connection to the named machine, harvest and hash the cert it gets back, save that cert as an exception, and then close the connection.

How does the user get the SHA3-256 hash of the cert to put on the command line?

Generate the fingerprint on the server side and securely distribute that to those in the organization who need to know it:

  $ openssl x509 -in CERT.pem -noout -sha256 -fingerprint

Note here that CERT.pem is the server's public key, not its private key.

For sites that don't care enough about security to bother with that, any common web browser should be able to produce this. For instance, in the current version of Firefox, the path is:

1. Click lock icon
2. Click flyout menu next to Firefox's "verdict"
3. On the Security Tab of the resulting Page Info dialog, click View Certificate
4. Scroll down to the Fingerprints section; Firefox currently gives both SHA-1 and SHA-256 fingerprints

The path varies by browser and by version.

This method is insecure because it allows MITM modification of the key, which the user may then add to a persistent trust store. (It's a sub-class of TOFU problem.)

Fingerprints tends to change much often that CA certs. Just my $0.02... ;-)

When the fingerprint changes, it's because the certificate changed, which means a Fossil user who insists on using self-signed server certificates has only two real choices here:

1. Go find the new fingerprint and give it to Fossil.

2. Don't use --server-fingerprint, and manually accept the changed certificate.

Yet another reason to use a certificate signed by a CA your system is configured to trust, either a publicly-trusted CA or a locally-configured one.