Jump to content

Optimization possible?


AdamS
 Share

Recommended Posts

Not sure if this is possible with Intella but I was hoping there might be a way to optimize Intella to make better use of the RAM and CPU.

 

I indexed 50gb of email archives yesterday and it took 20 hours all up, but checking system resources Intella was using less than 1% CPU and only 550mb of an available 12GB RAM.

 

Being a 32 application obviously it can't utilise all that RAM, but is there a way to tell it to use more RAM/CPU to speed the indexing up?

Link to comment
Share on other sites

Thanks Kathleen I will look at some of the issues that I can address easily, such as hard disk configuration and BIOS, however the flyer mentions RAM of 2GB minimum and 4GB recommended. I have 12GB and Intella is only using 550MB. So RAM would not really seem to be an issue in the indexing stage is that correct?

 

Edit : I have enable AHCI, ensured source and data files reside on seperate hard drives and have put the source and target folders into an 'ignore' list for the AV. I'm running indexing on the same 50GB archive now and will come back and see what the time difference is after these small changes.

Link to comment
Share on other sites

Okay, indexing finished overnight, down to 16 hours for the same data set.

 

A significant improvement but still about 10 hours slower than another tool indexing the same type of data set.

 

Kathleen beyond the steps mentioned in the document are there any other ways to force Intella to use more memory? I will look at some raided SSD drives as another possible solution but if there is a switch I can add to the executable or something like that...or am I grasping at straws?

 

Other than the slow indexing time I'm incredibly happy with the switch to Intella :)

Link to comment
Share on other sites

Hi Adam,

 

I have a i7 with 8 gig of ram and two sata disks. I would not expect 50 gig of PST's to take more than 5-8 hours. 16 hours is at least as twice as slow as I would expect.

 

One of the big four customers bench-marked Intella against other indexing tools and we came out ahead on the majority of test. So the speed should not be that noticeable if at all.

 

Do you have any AV or similar that may be causing interference.

 

Do you have a second PC you could perhaps try his on as a test? Intella was designed to run best on a mid range PC so our users could have good results on standard PC's. So it does not need to be a rocketship for the test.

 

As far as using more RAM this will not speed up Intella. By far the biggest issue is disk access with indexing tools. If you use more RAM you just create a bigger bottleneck for the disks.

Link to comment
Share on other sites

Hi Admin, I'm running an i7 CPU with 12gb Ram, there are 4 SATA HDD on the system, 1 for the OS and 3 for data storage.

 

What you are saying about the disks being the most likely bottleneck makes me think it may be the type of HDD I'm using. Out of habit I've put WD Green hard drives in the system, not thinking about performance but more about saving a few dollars for the boss ;)

 

I know the green drives are quite notorious for performance issues so I think I'll go and grab some performance drives and run the test again.

 

With regards to the AV I put the source and target folders in the ignore list for the AV, but I will run the scan again with AV completely disabled just to check.

 

A question regarding the HDD configuration. If I attach some faster drives to the exsisting system, one for the evidence data and one for the case data, will it matter that the OS is installed on a slower WD green drive? I'm trying to avoid a full reinstall if I can.

Link to comment
Share on other sites

Hello Adam,

 

For Intella processing the WD Green drives are certainly not ideal. They are designed to offer low cost per GB, low power consumption and low noise. They can be perfect for storing your digital media or to use as a backup medium, but for typical database-like access the WD Black disks will deliver much better performance. See e.g. the conclusion of this comparison for some insights: http://www.legitrevi...rticle/1077/10/. With Google you can easily find more reviews like these.

 

Furthermore it can be beneficial to google the exact disks that you use. Searching for WD Green I found e.g. this: http://forums.whirlp...archive/1367904. It suggests that the disk head is parked after 8 seconds of inactivity. That could mean that after processing each complex PDF, one or more disks need to wake up, which hurts performance considerably. Regardless of whether this particular page applies to your WD Green disks, this is the type of issue to look for when optimizing hardware that is not already designed for performance.

 

I think that keeping your Windows installation on a WD Green disk won't impact Intella processing much overall, but who knows. Perhaps the temp folder or the page file is still critical.

 

As for installed RAM: it is true that Intella as a 32-bit application can only use a limited amount of RAM, but note that during indexing multiple processes are typically active, whose combined use can go above the 32-bit limitations if necessary. Also, any unused RAM can be used by Windows as a file system cache. We have anecdotal evidence of indexing getting faster when expanding a 4 GB machine to 6 or 8 GB. With 12 GB already on-board I don't expect adding even more memory will result in any further improvements though.

 

Finally, it may be worth looking into what memory chips are installed and if they are placed according to the instructions that came with the motherboard. Nowadays dual and triple channel memory chips promise greater bandwidth, but these typically come with specific instructions on which memory banks to use in order for multi-channel access to work.

 

Let us know how your experiments go, we can all benefit from this!

Link to comment
Share on other sites

Thanks Chris, I think my first step is to go and get a couple of SSD drives and possibly a WD Black for the OS, I can mirror the OS drive across using a forensic tool to avoid the need for a reinstall. The memory I have is running in dual mode but is only medium quality, however my experience gaming and building rigs over the years leads me to believe the improvement for upgrading to high quality ram is usually quite small and not really cost effective.

 

If I can get the indexing of that data set down to under 10 hours I'll be happy, and the boss will be happy.

 

I'll report back after with the results from changing the HDD's.

Link to comment
Share on other sites

 Share

×
×
  • Create New...