Intella 1.8 beta 1

[Chris]

Hello all,

 

I am proud to announce the first beta of Intella 1.8. This version features many enhancements. A list of the most prominent ones:

• Greatly improved indexing speed; up to three times faster indexing time, depending on data set and hardware.
• Paragraph features: collapse/expand paragraphs, search for selected paragraphs in other items, optionally hide already seen paragraphs.
• Task wizard, for defining common processing steps such as applying a keyword list and tagging or exporting the results. These can be run automatically once indexing has completed or by manually starting them.
• Improved disk image processing: faster opening and processing, added support for MacOS, GPT and ISO images.
• The new Sets display shows how search results compare in size, without showing the overlaps between the results.
• The Contents tab now captures the original layout better, e.g. tables, font sizes and font styles are shown.
• Redaction refinements: added Delete button and Undo/Redo support.
• Prevent complex Cluster Maps from freezing the application.
• Added support for Bloomberg email dumps.
• A critical fix for Intella's dongle driver, necessary for users running on Windows 8.1.

Forum users who would like to test the new release are invited to reply to this topic or send me a private message. I will reply with a private message with instructions.

 

We look forward to your feedback! In particular we would like to hear about indexing performance stats: what type of size of data set did you index, how long did it take, what did you see in the Windows Resource Monitor, etc.

 

 

[admin]

See this link for the Optimization Folder usage. 

 

http://community.vound-software.com/index.php?/topic/185-beta-optimization-folder/

[Kalin]

Yep, let me try it, probably next week.

[AdamS]

Has the beta been locked from opening cases that were produced with 1.7.3 and vice versa?

[Chris]

Hello Adam,

 

That is correct. At the moment the 1.8 beta can only open cases made with it, 1.7.3 cases will be listed as disabled. This is due to the many database changes that were necessary for the improved indexing performance. The final version will have that functionality.

[jasoncovey]

Now that I have had a chance to get everything up and running with 1.8 beta, and getting all of my systems configured at a new employer, I wanted to report my results.  I believe they are on the spectacular side, having used Intella since 2011 at my prior job.

 

We have Intella Pro and Connect running in a 100% virtual environment, which is a major change for me.  Based on my prior knowledge and experience with Intella and its system requirements, I assumed this would be a complete disaster.  Not that case!

 

We built a Windows 7 Enterprise VM, with a 250 GB system drive, and 500 GB drives for Case Index and Case Data.  We later added a 500 GB drive for optimization, per Vound's specs.  The drives were segregated for use only with this VM, and are located in a SAN, connected via 8 GB fiber channel.  These drive sizes are all expandable, so these sizes were just specified for testing.  The VM lives on a Fujitsu server with massive RAM (256 GB?), to which it is specifically tethered, and is the least-used of a bank of 4.  The VM has 16 GB RAM assigned.  The dongle is mapped to the VM from this server.

 

Initially, we only had a single Xeon processor with 4 cores.  We went from 1 to 2 and then to 4 cores when we saw extremely high CPU usage.  The thinking was that we didn't want to ask VMWare to do too much.  Ultimately, we added a second, 4-core Xeon processor.  However, we still see very high system usage, which we have since been advised is a very good thing in 1.8.  We have also seen apparent RAM usage that is much lower than what I was used to, about which Chris advised us is only part of the story, as the remainder of the RAM is being used for disk caching, which is increasing indexing performance.

 

Although I haven't been able to perform a perfect apples to apples comparison in every single respect, I ran the same indexing job on the same data set.  It consisted of a relatively small amount of foldered data, as well as approx. 397 PSTs of varying size from tiny to 4 GB.  There were probably 80+ that were 1 GB or larger. 

 

The results are posted below.  Note that 1.8 is indexing items within items even though I would prefer it not, and 1.7.3 was not.  Still, I think the results will speak for themselves.

 

Before:

 

 

After:

 

 

That's 72.5% faster by my math. 

 

Previously, on a Win 2008 R2 physical server with 16 GB RAM and a single, slower processor but dedicated internal drives, I was thrilled for any clock cycle that maintained 2K items per minute, and frequently had to settle for 1K.

 

Hopefully that will give some of you a better idea of some real world expectations with regard to indexing with 1.8. 

 

[balzanto]

I'm running Beta 1 over a folder containing 56 PST/OST files totaling approximately 180GB.  Individual file sizes are between <1GB through 7-10GB.  I've configured as recommended with the Intella application, the case, the evidence and the temp files on separate physical drives.  Intella is processing much faster than 1.7.3 but crashes prior to finishing.  When I say crash, I mean that when I come back to check on it, Intella is no longer running.  It could be a bad PST file but that's hard to figure out given the size of the log files (over 100MB).

 

One more issue I had was HASP errors terminating Intella Beta 1.  I updated to the latest HASP drivers from their site and so far (hope), that hasn't come up again.

[admin]

I'm running Beta 1 over a folder containing 56 PST/OST files totaling approximately 180GB.  Individual file sizes are between

 

One more issue I had was HASP errors terminating Intella Beta 1.  I updated to the latest HASP drivers from their site and so far (hope), that hasn't come up again.

 

 

Hi - I know that we gave you a pre-beta version. I will PM you the link to the beta 1. It should solve the issue. 

[Chris]

Intella is processing much faster than 1.7.3 but crashes prior to finishing.  When I say crash, I mean that when I come back to check on it, Intella is no longer running.  It could be a bad PST file but that's hard to figure out given the size of the log files (over 100MB).

 

When this keeps happening with beta 1, could you share the logs with us via a support ticket?

[Chris]

The results are posted below.  Note that 1.8 is indexing items within items even though I would prefer it not, and 1.7.3 was not.  Still, I think the results will speak for themselves.

 

Many thanks for sharing! The 1.8 beta indeed seems to ignore the embedded items setting, it simply always extracts all embedded items. We will make sure that gets solved in the next beta. Note that that will make the total indexing duration with 1.8 even shorter, as now it is unnecessarily doing more than 1.7.3 did!

[jasoncovey]

I just ran a new job last night, and achieved similarly-spectacular results, so I wanted to share.  This was for a 129 GB data set, culled from a 1.4 TB collection by file extension, maintaining source folder structures.  60 GB of PSTs, 43 of PDFs, and the rest divided among common document and loose email file types.  

 

The job completed in just under 10 hours.  This includes the extraction of over 1.1 millions TIFF images from the PDFs, which is never desired in the context with which I use Intella.  So, like Chris said, it will probably be faster when not doing all the unnecessary processing (in my case). 

 

Looking forward to the official release!

 

 

 

[Chris]

Hi Jason and others,

 

The second beta has just been released, see http://community.vound-software.com/index.php?/topic/202-intella-18-beta-2/