Skip to content

Commit

Permalink
[PATCH] Improve git-rev-list memory usage further
Browse files Browse the repository at this point in the history
This avoids keeping tree entries around, and free's them as it traverses
the list. This avoids building up a huge memory footprint just for these
small but very common allocations.

Before:

	$ /usr/bin/time git-rev-list --objects v2.6.12..HEAD | wc -l
	11.65user 0.38system 0:12.65elapsed 95%CPU (0avgtext+0avgdata 0maxresident)k
	0inputs+0outputs (0major+42934minor)pagefaults 0swaps
	59124

After:

	$ /usr/bin/time git-rev-list --objects v2.6.12..HEAD | wc -l
	12.28user 0.29system 0:12.57elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
	0inputs+0outputs (0major+26718minor)pagefaults 0swaps
	59124

Note how the minor fault numbers - which ends up being how many pages we
needed to map - go down from 42934 (167 MB) to 26718 (104 MB).  That is:

Before:
	42934 minor pagefaults

After:

	26718 minor pagefaults

This is all in _addition_ to the previous fixes.  It used to be
~48,000 pagefaults.

That's still a honking big memory footprint, but it's about half of what
it was just a day or two ago (and this is the object list for a pretty big
update - almost 60,000 objects. Smaller updates need less memory).

Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
  • Loading branch information
Linus Torvalds authored and Junio C Hamano committed Sep 16, 2005
1 parent d0ac30f commit b0d8923
Showing 1 changed file with 10 additions and 2 deletions.
12 changes: 10 additions & 2 deletions rev-list.c
Original file line number Diff line number Diff line change
Expand Up @@ -147,11 +147,16 @@ static struct object_list **process_tree(struct tree *tree, struct object_list *
die("bad tree object %s", sha1_to_hex(obj->sha1));
obj->flags |= SEEN;
p = add_object(obj, p, name);
for (entry = tree->entries ; entry ; entry = entry->next) {
entry = tree->entries;
tree->entries = NULL;
while (entry) {
struct tree_entry_list *next = entry->next;
if (entry->directory)
p = process_tree(entry->item.tree, p, entry->name);
else
p = process_blob(entry->item.blob, p, entry->name);
free(entry);
entry = next;
}
return p;
}
Expand Down Expand Up @@ -218,12 +223,15 @@ static void mark_tree_uninteresting(struct tree *tree)
if (parse_tree(tree) < 0)
die("bad tree %s", sha1_to_hex(obj->sha1));
entry = tree->entries;
tree->entries = NULL;
while (entry) {
struct tree_entry_list *next = entry->next;
if (entry->directory)
mark_tree_uninteresting(entry->item.tree);
else
mark_blob_uninteresting(entry->item.blob);
entry = entry->next;
free(entry);
entry = next;
}
}

Expand Down

0 comments on commit b0d8923

Please sign in to comment.