Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
Comment: | Reducing the leak of annotate even more.
I wrote a TODO in the file, for a leak still present and relevant for files modified by many versions, but that requires harder work to get right. |
---|---|
Downloads: | Tarball | ZIP archive |
Timelines: | family | ancestors | descendants | both | annotate_noleak |
Files: | files | file ages | folders |
SHA1: |
e2ebb1f5cae83e85a03bf4a32a48bb13 |
User & Date: | viriketo 2011-09-04 21:39:49.384 |
2011-09-04
| ||
22:14 |
Implementation of a linked list to solve the memory leak described in a TODO
in [e2ebb1f5cae8].
This code is slower than having the memory leak, and at the end, it was not a big memory leak. Let's say, 10 byte per revision involved in a file annotate. If a file has 30000 revisions, it may go around 300KB then. For this leak to be noticeable (as we have a content cache until 50MB), it would mean having above a milion revisions for the file annotated. So, I keep this in a branch apart, as maybe the leak is worth the faster code. I have not measured the speed difference though. ... (check-in: ef8266b7 user: viriketo tags: label_linkedlist) | |
2011-09-13
| ||
16:43 | Getting in the changes reducing the leak on annotate. In the fossil mailing list, I received only opinions in favour of getting mallocs and frees properly paired, so I think noone will refuse the change. ... (check-in: 7870a89b user: viriketo tags: annotate_noleak) | |
2011-09-04
| ||
22:14 |
Implementation of a linked list to solve the memory leak described in a TODO
in [e2ebb1f5cae8].
This code is slower than having the memory leak, and at the end, it was not a big memory leak. Let's say, 10 byte per revision involved in a file annotate. If a file has 30000 revisions, it may go around 300KB then. For this leak to be noticeable (as we have a content cache until 50MB), it would mean having above a milion revisions for the file annotated. So, I keep this in a branch apart, as maybe the leak is worth the faster code. I have not measured the speed difference though. ... (check-in: ef8266b7 user: viriketo tags: label_linkedlist) | |
21:39 |
Reducing the leak of annotate even more.
I wrote a TODO in the file, for a leak still present and relevant for files modified by many versions, but that requires harder work to get right. ... (check-in: e2ebb1f5 user: viriketo tags: annotate_noleak) | |
21:02 | Making annotate not segfault for the complex test case I have. There is still a little leak... but muuuch smaller than it was. ... (check-in: fa760a24 user: viriketo tags: annotate_noleak) | |
︙ | ︙ | |||
635 636 637 638 639 640 641 642 643 644 645 646 647 648 | const char *zSrc; /* Tag showing origin of this line */ } *aOrig; int nOrig; /* Number of elements in aOrig[] */ int nNoSrc; /* Number of entries where aOrig[].zSrc==NULL */ int iLevel; /* Current level */ int nVers; /* Number of versions analyzed */ char **azVers; /* Names of versions analyzed */ }; /* ** Initialize the annotation process by specifying the file that is ** to be annotated. The annotator takes control of the input Blob and ** will release it when it is finished with it. */ | > | < | > | 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 | const char *zSrc; /* Tag showing origin of this line */ } *aOrig; int nOrig; /* Number of elements in aOrig[] */ int nNoSrc; /* Number of entries where aOrig[].zSrc==NULL */ int iLevel; /* Current level */ int nVers; /* Number of versions analyzed */ char **azVers; /* Names of versions analyzed */ Blob toAnnotate; }; /* ** Initialize the annotation process by specifying the file that is ** to be annotated. The annotator takes control of the input Blob and ** will release it when it is finished with it. */ static int annotation_start(Annotator *p){ int i; p->c.aTo = break_into_lines(blob_str(&p->toAnnotate), blob_size(&p->toAnnotate),&p->c.nTo,1); if( p->c.aTo==0 ){ return 1; } p->aOrig = fossil_malloc( sizeof(p->aOrig[0])*p->c.nTo ); for(i=0; i<p->c.nTo; i++){ p->aOrig[i].z = p->c.aTo[i].z; p->aOrig[i].n = p->c.aTo[i].h & LENGTH_MASK; |
︙ | ︙ | |||
677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 | int iPrevLevel; int iThisLevel; /* Prepare the parent file to be diffed */ p->c.aFrom = break_into_lines(blob_str(pParent), blob_size(pParent), &p->c.nFrom, 1); if( p->c.aFrom==0 ){ return 1; } /* Compute the differences going from pParent to the file being ** annotated. */ diff_all(&p->c); /* Where new lines are inserted on this difference, record the ** zPName as the source of the new line. */ iPrevLevel = p->iLevel; p->iLevel++; iThisLevel = p->iLevel; for(i=lnTo=0; i<p->c.nEdit; i+=3){ struct AnnLine *x = &p->aOrig[lnTo]; for(j=0; j<p->c.aEdit[i]; j++, lnTo++, x++){ if( x->zSrc==0 || x->iLevel==iPrevLevel ){ x->zSrc = zPName; x->iLevel = iThisLevel; } } lnTo += p->c.aEdit[i+2]; } | > > > | 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 | int iPrevLevel; int iThisLevel; /* Prepare the parent file to be diffed */ p->c.aFrom = break_into_lines(blob_str(pParent), blob_size(pParent), &p->c.nFrom, 1); if( p->c.aFrom==0 ){ free(p->c.aFrom); return 1; } /* Compute the differences going from pParent to the file being ** annotated. */ diff_all(&p->c); /* Where new lines are inserted on this difference, record the ** zPName as the source of the new line. */ iPrevLevel = p->iLevel; p->iLevel++; iThisLevel = p->iLevel; for(i=lnTo=0; i<p->c.nEdit; i+=3){ struct AnnLine *x = &p->aOrig[lnTo]; for(j=0; j<p->c.aEdit[i]; j++, lnTo++, x++){ if( x->zSrc==0 || x->iLevel==iPrevLevel ){ /* TODO: handle zPName so we can free labels * if they get totally unreferenced in the Annotator */ x->zSrc = zPName; x->iLevel = iThisLevel; } } lnTo += p->c.aEdit[i+2]; } |
︙ | ︙ | |||
720 721 722 723 724 725 726 | } /* ** COMMAND: test-annotate-step */ void test_annotate_step_cmd(void){ | | > | | | | 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 | } /* ** COMMAND: test-annotate-step */ void test_annotate_step_cmd(void){ Blob b = empty_blob; Annotator x; int i; if( g.argc<4 ) usage("RID1 RID2 ..."); db_must_be_within_tree(); memset(&x, 0, sizeof(x)); x.toAnnotate = empty_blob; content_get(name_to_rid(g.argv[2]), &x.toAnnotate); if( annotation_start(&x) ){ fossil_fatal("binary file"); } for(i=3; i<g.argc; i++){ blob_zero(&b); content_get(name_to_rid(g.argv[i]), &b); if( annotation_step(&x, &b, g.argv[i-1]) ){ fossil_fatal("binary file"); |
︙ | ︙ | |||
761 762 763 764 765 766 767 | Annotator *p, /* The annotator */ int fnid, /* The name of the file to be annotated */ int mid, /* Use the version of the file in this check-in */ int webLabel, /* Use web-style annotations if true */ int iLimit, /* Limit the number of levels if greater than zero */ int annFlags /* Flags to alter the annotation */ ){ | < > > | | | 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 | Annotator *p, /* The annotator */ int fnid, /* The name of the file to be annotated */ int mid, /* Use the version of the file in this check-in */ int webLabel, /* Use web-style annotations if true */ int iLimit, /* Limit the number of levels if greater than zero */ int annFlags /* Flags to alter the annotation */ ){ Blob step = empty_blob; /* Text of previous revision */ int rid; /* Artifact ID of the file being annotated */ char *zLabel; /* Label to apply to a line */ Stmt q; /* Query returning all ancestor versions */ /* Initialize the annotation */ rid = db_int(0, "SELECT fid FROM mlink WHERE mid=%d AND fnid=%d",mid,fnid); if( rid==0 ){ fossil_panic("file #%d is unchanged in manifest #%d", fnid, mid); } memset(p, 0, sizeof(*p)); p->toAnnotate = empty_blob; if( !content_get(rid, &p->toAnnotate) ){ fossil_panic("unable to retrieve content of artifact #%d", rid); } db_multi_exec("CREATE TEMP TABLE ok(rid INTEGER PRIMARY KEY)"); if( iLimit<=0 ) iLimit = 1000000000; compute_direct_ancestors(mid, iLimit); annotation_start(p); db_prepare(&q, "SELECT mlink.fid," " (SELECT uuid FROM blob WHERE rid=mlink.%s)," " date(event.mtime), " " coalesce(event.euser,event.user) " " FROM ancestor, mlink, event" |
︙ | ︙ | |||
816 817 818 819 820 821 822 823 824 825 826 827 828 829 | p->azVers = fossil_realloc(p->azVers, p->nVers*sizeof(p->azVers[0]) ); p->azVers[p->nVers-1] = zLabel; content_get(pid, &step); annotation_step(p, &step, zLabel); blob_reset(&step); } db_finalize(&q); } /* ** WEBPAGE: annotate ** ** Query parameters: ** | > | 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 | p->azVers = fossil_realloc(p->azVers, p->nVers*sizeof(p->azVers[0]) ); p->azVers[p->nVers-1] = zLabel; content_get(pid, &step); annotation_step(p, &step, zLabel); blob_reset(&step); } db_finalize(&q); free(p->c.aTo); } /* ** WEBPAGE: annotate ** ** Query parameters: ** |
︙ | ︙ | |||
912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 | if( fnid==0 ){ fossil_fatal("no such file: %s", zFilename); } fid = db_int(0, "SELECT rid FROM vfile WHERE pathname=%Q", zFilename); if( fid==0 ){ fossil_fatal("not part of current checkout: %s", zFilename); } mid = db_int(0, "SELECT mid FROM mlink WHERE fid=%d AND fnid=%d", fid, fnid); if( mid==0 ){ fossil_panic("unable to find manifest"); } if( fileVers ) annFlags |= ANN_FILE_VERS; annotate_file(&ann, fnid, mid, 0, iLimit, annFlags); if( showLog ){ for(i=0; i<ann.nVers; i++){ printf("version %3d: %s\n", i+1, ann.azVers[i]); } printf("---------------------------------------------------\n"); } for(i=0; i<ann.nOrig; i++){ fossil_print("%s: %.*s\n", ann.aOrig[i].zSrc, ann.aOrig[i].n, ann.aOrig[i].z); } } | > > > > | 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 | if( fnid==0 ){ fossil_fatal("no such file: %s", zFilename); } fid = db_int(0, "SELECT rid FROM vfile WHERE pathname=%Q", zFilename); if( fid==0 ){ fossil_fatal("not part of current checkout: %s", zFilename); } blob_reset(&treename); mid = db_int(0, "SELECT mid FROM mlink WHERE fid=%d AND fnid=%d", fid, fnid); if( mid==0 ){ fossil_panic("unable to find manifest"); } if( fileVers ) annFlags |= ANN_FILE_VERS; annotate_file(&ann, fnid, mid, 0, iLimit, annFlags); if( showLog ){ for(i=0; i<ann.nVers; i++){ printf("version %3d: %s\n", i+1, ann.azVers[i]); } printf("---------------------------------------------------\n"); } for(i=0; i<ann.nOrig; i++){ fossil_print("%s: %.*s\n", ann.aOrig[i].zSrc, ann.aOrig[i].n, ann.aOrig[i].z); } free(ann.azVers); free(ann.aOrig); blob_reset(&ann.toAnnotate); } |