Opened 13 years ago

Last modified 8 years ago

#29 assigned defect

'search -r' is very slow

Reported by: baniszew Owned by: nelhage@MIT.EDU
Priority: minor Milestone:
Component: ui Keywords:
Cc:

Description (last modified by nelhage@MIT.EDU)

"search -r" is far slower than e.g. filtering using a custom filter on the body. The root cause is that searching renders each message, which results in a callout to perl, rather than just staying in C and examining fields. I suspect the right answer is to define search to search over (a fixed set of? all?) message fields, rather than the rendered body.

Change History (8)

comment:1 Changed 13 years ago by asedeno@MIT.EDU

This is due to BarnOwl needing to render (in memory, not on screen) all messages from the current message back to the first message as it searches. I'm curious to see how this churns the fmtext cache.

comment:2 Changed 13 years ago by nelhage@MIT.EDU

Do we actually know that this is due to the rendering, or because it has to swap all the messages into memory, or what? I'd be curious to see some profiling.

comment:3 Changed 13 years ago by nelhage@MIT.EDU

search -r apparently eats _insane_ amounts of memory. I suspect this is somehow the fmtext cache's fault, but I don't know. I need to do some debugging.

comment:4 Changed 13 years ago by nelhage@MIT.EDU

  • Component changed from zephyr to ui

I can't find any leaks ... I think we may just be seeing horrible heap thrashing by all the back-and-forth between C and perl. I'm going to hope asedeno's utf-8 fmtext fixes mean we can ditch the fmtext cache and won't have this problem as much...

comment:5 Changed 12 years ago by nelhage@MIT.EDU

  • Priority changed from minor to major
  • Status changed from new to assigned

Bleh. There sure seems to be a memory leak here. I'll try to look into it.

comment:6 Changed 12 years ago by andersk@MIT.EDU

  • Priority changed from major to minor

The memory leak is fixed in commit 1639ff733f7f12836c6bcf7bbf77a9cf4d225b10 and commit e574a0048546c4051f73d18b63169008df045689. search isn’t significantly faster now, but it is a hell of a lot less dangerous.

comment:7 Changed 12 years ago by nelhage@MIT.EDU

  • Description modified (diff)
  • Summary changed from getting stuck on search -r to 'search -r' is very slow

comment:8 Changed 8 years ago by geofft@mit.edu

Is this specific to search -r, or is this just about the fact that formatting messages for searching (either forwards or backwards) is slow?

Note: See TracTickets for help on using tickets.