Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
Overview
Comment: | Improvements to the way backoffice is launched, especially on unix where it now runs in a separate process using fork(). Also fix some minor bugs in other parts of the system that were found while testing backoffice. |
---|---|
Downloads: | Tarball | ZIP archive |
Timelines: | family | ancestors | descendants | both | trunk |
Files: | files | file ages | folders |
SHA3-256: |
4180dc6b5a6c226fd22c8b3551dcd26b |
User & Date: | drh 2018-08-07 18:30:15.898 |
Context
2018-08-07
| ||
18:53 | Allow manifest artifacts to omit the C and U cards, because otherwise there are some historical manifest artifacts in Fossil itself that will not parse, and there may be similar artifacts in other repositories. ... (check-in: b3ccc4bf user: drh tags: trunk) | |
18:30 | Improvements to the way backoffice is launched, especially on unix where it now runs in a separate process using fork(). Also fix some minor bugs in other parts of the system that were found while testing backoffice. ... (check-in: 4180dc6b user: drh tags: trunk) | |
18:28 | Improvements to comments. No code changes. ... (Closed-Leaf check-in: 1b54dd79 user: drh tags: fork-backoffice) | |
15:12 | A new implementation for "Forum" in which each forum post is an artifact. This merge includes lots of enhancements to email notification, backoffice, configuration, and other subsystems, all in support of the new forum artifacts. The forum feature is not complete nor bug-free but at this point it seems good enough to continue development on trunk. ... (check-in: 99fcc43f user: drh tags: trunk) | |
Changes
Changes to src/backoffice.c.
︙ | ︙ | |||
33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 | ** ** At the same time, we do not want a backoffice process to run forever. ** Backoffice processes should die off after doing whatever work they need ** to do. In this way, we avoid having lots of idle processes in the ** process table, doing nothing on rarely accessed repositories, and ** if the Fossil binary is updated on a system, the backoffice processes ** will restart using the new binary automatically. */ #include "config.h" #include "backoffice.h" #include <time.h> #if defined(_WIN32) # include <windows.h> #else # include <unistd.h> # include <sys/types.h> # include <signal.h> #endif /* ** The BKOFCE_LEASE_TIME is the amount of time for which a single backoffice ** processing run is valid. Each backoffice run monopolizes the lease for ** at least this amount of time. Hopefully all backoffice processing is ** finished much faster than this - usually in less than a second. But | > > > > > > > > > > > > > > > > > > > | | | | | | | > > | | > > > > > > > > > > > > | > > > > > > | 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 | ** ** At the same time, we do not want a backoffice process to run forever. ** Backoffice processes should die off after doing whatever work they need ** to do. In this way, we avoid having lots of idle processes in the ** process table, doing nothing on rarely accessed repositories, and ** if the Fossil binary is updated on a system, the backoffice processes ** will restart using the new binary automatically. ** ** At any point in time there should be at most two backoffice processes. ** There is a main process that is doing the actually work, and there is ** a second stand-by process that is waiting for the main process to finish ** and that will become the main process after a delay. ** ** After any successful web page reply, the backoffice_check_if_needed() ** routine is called. That routine checks to see if both one or both of ** the backoffice processes are already running. That routine remembers the ** status in a global variable. ** ** Later, after the repository database is closed, the ** backoffice_run_if_needed() routine is called. If the prior call ** to backoffice_check_if_needed() indicated that backoffice processing ** might be required, the run_if_needed() attempts to kick off a backoffice ** process. ** ** All work performance by the backoffice is in the backoffice_work() ** routine. */ #include "config.h" #include "backoffice.h" #include <time.h> #if defined(_WIN32) # include <windows.h> #else # include <unistd.h> # include <sys/types.h> # include <signal.h> #endif /* ** The BKOFCE_LEASE_TIME is the amount of time for which a single backoffice ** processing run is valid. Each backoffice run monopolizes the lease for ** at least this amount of time. Hopefully all backoffice processing is ** finished much faster than this - usually in less than a second. But ** regardless of how long each invocation lasts, successive backoffice runs ** must be spaced out by at least this much time. */ #define BKOFCE_LEASE_TIME 60 /* Length of lease validity in seconds */ #if LOCAL_INTERFACE /* ** An instance of the following object describes a lease on the backoffice ** processing timeslot. This lease is used to help ensure that no more than ** one process is running backoffice at a time. */ struct Lease { sqlite3_uint64 idCurrent; /* process ID for the current lease holder */ sqlite3_uint64 tmCurrent; /* Expiration of the current lease */ sqlite3_uint64 idNext; /* process ID for the next lease holder on queue */ sqlite3_uint64 tmNext; /* Expiration of the next lease */ }; #endif /*************************************************************************** ** Local state variables ** ** Set to prevent backoffice processing from ever entering sleep or ** otherwise taking a long time to complete. Set this when a user-visible ** process might need to wait for backoffice to complete. */ static int backofficeNoDelay = 0; /* This variable is set to the name of a database on which backoffice ** should run if backoffice process is needed. It is set by the ** backoffice_check_if_needed() routine which must be run while the database ** file is open. Later, after the database is closed, the ** backoffice_run_if_needed() will consult this variable to see if it ** should be a no-op. */ static char *backofficeDb = 0; /* End of state variables ****************************************************************************/ /* ** Do not allow backoffice processes to sleep waiting on a timeslot. ** They must either do their work immediately or exit. ** ** In a perfect world, this interface would not exist, as there would ** never be a problem with waiting backoffice threads. But in some cases ** a backoffice will delay a UI thread, so we don't want them to run for ** longer than needed. */ void backoffice_no_delay(void){ backofficeNoDelay = 1; } /* ** Parse a unsigned 64-bit integer from a string. Return a pointer |
︙ | ︙ | |||
128 129 130 131 132 133 134 | "REPLACE INTO repository.config(name,value,mtime)" " VALUES('backoffice','%lld %lld %lld %lld',now())", pLease->idCurrent, pLease->tmCurrent, pLease->idNext, pLease->tmNext); } /* | | | | 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 | "REPLACE INTO repository.config(name,value,mtime)" " VALUES('backoffice','%lld %lld %lld %lld',now())", pLease->idCurrent, pLease->tmCurrent, pLease->idNext, pLease->tmNext); } /* ** Check to see if the process identified by pid is alive. If ** we cannot prove the the process is dead, return true. */ static int backofficeProcessExists(sqlite3_uint64 pid){ #if defined(_WIN32) return 1; #else return pid>0 && kill((pid_t)pid, 0)==0; #endif } /* ** Check to see if the process identified by pid has finished. If ** we cannot prove the the process is still running, return true. */ static int backofficeProcessDone(sqlite3_uint64 pid){ #if defined(_WIN32) return 1; #else return pid<=0 || kill((pid_t)pid, 0)!=0; |
︙ | ︙ | |||
203 204 205 206 207 208 209 | sqlite3_uint64 x = (sqlite3_uint64)atoi(g.argv[i]); fossil_print("ProcessId %lld: exists %d done %d\n", x, backofficeProcessExists(x), backofficeProcessDone(x)); } } | > | | | < < < | | > > | | | > > > > > > > | > > > > > > | | | > > > > > > > > > > > > > > > > > > > > > > > | 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 | sqlite3_uint64 x = (sqlite3_uint64)atoi(g.argv[i]); fossil_print("ProcessId %lld: exists %d done %d\n", x, backofficeProcessExists(x), backofficeProcessDone(x)); } } /* ** If backoffice processing is needed set the backofficeDb variable to the ** name of the database file. If no backoffice processing is needed, ** this routine makes no changes to state. */ void backoffice_check_if_needed(void){ Lease x; sqlite3_uint64 tmNow; if( backofficeDb ) return; if( g.zRepositoryName==0 ) return; if( g.db==0 ) return; tmNow = time(0); backofficeReadLease(&x); if( x.tmNext>=tmNow && backofficeProcessExists(x.idNext) ){ /* Another backoffice process is already queued up to run. This ** process does not need to do any backoffice work. */ return; }else{ /* We need to run backup to be (at a minimum) on-deck */ backofficeDb = fossil_strdup(g.zRepositoryName); } } /* ** Check for errors prior to running backoffice_thread() or backoffice_run(). */ static void backoffice_error_check_one(int *pOnce){ if( *pOnce ){ fossil_panic("multiple calls to backoffice()"); } *pOnce = 1; if( g.db==0 ){ fossil_panic("database not open for backoffice processing"); } if( db_transaction_nesting_depth()!=0 ){ fossil_panic("transaction %s not closed prior to backoffice processing", db_transaction_start_point()); } } /* This is the main loop for backoffice processing. ** ** If another process is already working as the current backoffice and ** the on-deck backoffice, then this routine returns very quickly ** without doing any work. ** ** If no backoffice processes are running at all, this routine becomes ** the main backoffice. ** ** If a primary backoffice is running, but a on-deck backoffice is ** needed, this routine becomes that on-desk backoffice. */ static void backoffice_thread(void){ Lease x; sqlite3_uint64 tmNow; sqlite3_uint64 idSelf; int lastWarning = 0; int warningDelay = 30; static int once = 0; backoffice_error_check_one(&once); backofficeTimeout(BKOFCE_LEASE_TIME*2); idSelf = backofficeProcessId(); while(1){ tmNow = time(0); db_begin_write(); backofficeReadLease(&x); if( x.tmNext>=tmNow |
︙ | ︙ | |||
301 302 303 304 305 306 307 | ** backoffice processing tasks, add them here. */ void backoffice_work(void){ email_backoffice(0); } /* | | | | > | > | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 | ** backoffice processing tasks, add them here. */ void backoffice_work(void){ email_backoffice(0); } /* ** COMMAND: backoffice ** ** Usage: backoffice [-R repository] ** ** Run backoffice processing. This might be done by a cron job or ** similar to make sure backoffice processing happens periodically. */ void backoffice_command(void){ verify_all_options(); db_find_and_open_repository(0,0); backoffice_thread(); } /* ** This is the main interface to backoffice from the rest of the system. ** This routine launches either backoffice_thread() directly or as a ** subprocess. */ void backoffice_run_if_needed(void){ if( backofficeDb==0 ) return; if( strcmp(backofficeDb,"x")==0 ) return; if( g.db ) return; if( g.repositoryOpen ) return; #if !defined(_WIN32) { pid_t pid = fork(); if( pid>0 ){ /* This is the parent in a successful fork(). Return immediately. */ if( g.fAnyTrace ){ fprintf(stderr, "/***** Subprocess %d creates backoffice child %d *****/\n", getpid(), (int)pid); } return; } if( pid==0 ){ /* This is the child of a successful fork(). Run backoffice. */ db_open_repository(backofficeDb); backofficeDb = "x"; backoffice_thread(); db_close(1); if( g.fAnyTrace ){ fprintf(stderr, "/***** Backoffice Child %d exits *****/\n", getpid()); } exit(0); } } #endif /* Fork() failed or is unavailable. Run backoffice in this process, but ** do so with the no-delay setting. */ backofficeNoDelay = 1; db_open_repository(backofficeDb); backofficeDb = "x"; backoffice_thread(); db_close(1); } |
Changes to src/cgi.c.
︙ | ︙ | |||
342 343 344 345 346 347 348 | fflush(g.httpOut); CGIDEBUG(("DONE\n")); /* After the webpage has been sent, do any useful background ** processing. */ g.cgiOutput = 2; | | < < < < < < < < < | < | 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 | fflush(g.httpOut); CGIDEBUG(("DONE\n")); /* After the webpage has been sent, do any useful background ** processing. */ g.cgiOutput = 2; if( g.db!=0 && iReplyStatus==200 ){ backoffice_check_if_needed(); } } /* ** Do a redirect request to the URL given in the argument. ** ** The URL must be relative to the base of the fossil server. |
︙ | ︙ |
Changes to src/db.c.
︙ | ︙ | |||
1878 1879 1880 1881 1882 1883 1884 1885 1886 1887 1888 1889 1890 1891 | } g.db = 0; } g.repositoryOpen = 0; g.localOpen = 0; assert( g.dbConfig==0 ); assert( g.zConfigDbName==0 ); } /* ** Close the database as quickly as possible without unnecessary processing. */ void db_panic_close(void){ if( g.db ){ | > | 1878 1879 1880 1881 1882 1883 1884 1885 1886 1887 1888 1889 1890 1891 1892 | } g.db = 0; } g.repositoryOpen = 0; g.localOpen = 0; assert( g.dbConfig==0 ); assert( g.zConfigDbName==0 ); backoffice_run_if_needed(); } /* ** Close the database as quickly as possible without unnecessary processing. */ void db_panic_close(void){ if( g.db ){ |
︙ | ︙ |
Changes to src/forum.c.
︙ | ︙ | |||
942 943 944 945 946 947 948 949 950 | const char *zTitle = db_column_text(&q, 2); @ <tr><td>%h(zAge) ago</td> @ <td>%z(href("%R/forumpost/%S",zUuid))%h(zTitle)</a> @ </tr> fossil_free(zAge); } @ </table></div> style_footer(); } | > | 942 943 944 945 946 947 948 949 950 951 | const char *zTitle = db_column_text(&q, 2); @ <tr><td>%h(zAge) ago</td> @ <td>%z(href("%R/forumpost/%S",zUuid))%h(zTitle)</a> @ </tr> fossil_free(zAge); } @ </table></div> db_finalize(&q); style_footer(); } |
Changes to src/http_transport.c.
︙ | ︙ | |||
268 269 270 271 272 273 274 | ** it is time to being receiving a reply. */ void transport_flip(UrlData *pUrlData){ if( pUrlData->isFile ){ char *zCmd; fclose(transport.pFile); zCmd = mprintf("\"%s\" http --in \"%s\" --out \"%s\" --ipaddr 127.0.0.1" | | | 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 | ** it is time to being receiving a reply. */ void transport_flip(UrlData *pUrlData){ if( pUrlData->isFile ){ char *zCmd; fclose(transport.pFile); zCmd = mprintf("\"%s\" http --in \"%s\" --out \"%s\" --ipaddr 127.0.0.1" " \"%s\" --localauth", g.nameOfExe, transport.zOutFile, transport.zInFile, pUrlData->name ); fossil_system(zCmd); free(zCmd); transport.pFile = fossil_fopen(transport.zInFile, "rb"); } } |
︙ | ︙ |
Changes to src/printf.c.
︙ | ︙ | |||
1079 1080 1081 1082 1083 1084 1085 | int rc = 1; char z[1000]; static int once = 0; if( once ) exit(1); once = 1; mainInFatalError = 1; | | | 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 | int rc = 1; char z[1000]; static int once = 0; if( once ) exit(1); once = 1; mainInFatalError = 1; /* db_force_rollback(); */ va_start(ap, zFormat); sqlite3_vsnprintf(sizeof(z),z,zFormat, ap); va_end(ap); if( g.fAnyTrace ){ fprintf(stderr, "/***** panic on %d *****/\n", getpid()); } fossil_errorlog("panic: %s", z); |
︙ | ︙ |
Changes to src/winhttp.c.
︙ | ︙ | |||
403 404 405 406 407 408 409 | } fossil_free(zIp); aux = fossil_fopen(zCmdFName, "wb"); if( aux==0 ) goto end_request; fwrite(zCmd, 1, strlen(zCmd), aux); sqlite3_snprintf(sizeof(zCmd), zCmd, | | | 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 | } fossil_free(zIp); aux = fossil_fopen(zCmdFName, "wb"); if( aux==0 ) goto end_request; fwrite(zCmd, 1, strlen(zCmd), aux); sqlite3_snprintf(sizeof(zCmd), zCmd, "\"%s\" http -args \"%s\" --nossl%s", g.nameOfExe, zCmdFName, p->zOptions ); in = fossil_fopen(zReplyFName, "w+b"); fflush(out); fflush(aux); fossil_system(zCmd); if( in ){ |
︙ | ︙ | |||
474 475 476 477 478 479 480 | fwrite(zHdr, 1, got, out); wanted += got; } assert( g.zRepositoryName && g.zRepositoryName[0] ); zIp = SocketAddr_toString(&p->addr); sqlite3_snprintf(sizeof(zCmd), zCmd, "\"%s\" http --in \"%s\" --out \"%s\" --ipaddr %s \"%s\"" | | | 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 | fwrite(zHdr, 1, got, out); wanted += got; } assert( g.zRepositoryName && g.zRepositoryName[0] ); zIp = SocketAddr_toString(&p->addr); sqlite3_snprintf(sizeof(zCmd), zCmd, "\"%s\" http --in \"%s\" --out \"%s\" --ipaddr %s \"%s\"" " --scgi --nossl%s", g.nameOfExe, zRequestFName, zReplyFName, zIp, g.zRepositoryName, p->zOptions ); fossil_free(zIp); in = fossil_fopen(zReplyFName, "w+b"); fflush(out); fossil_system(zCmd); |
︙ | ︙ |