SCANPST vs Intella error correction

[Guest dk_mitch]

Hi everyone

 

I am working on a case where I had a reason to suspect that my original PST source files were corrupted somehow.

So I ran SCANPST on them and the new files noticeably larger.

 

In all the time I have used Intella v 1 days and over 500 cases since, I have only noticed 4 or 5 times from Intella that there was a potential issue with the PST / OST file and to use SCANPST to correct it.

 

In this situation , this warning did not take place and since we do several hundred cases each year, I had no reason to automatically run SCANPST on the PST files before creating an Intella case and indexing.

 

So I would like to ask the ADMIN's to explain what is done while Intella first ingests / crawls a pst file.

Secondly, what have the members of this forum encountered after using SCANPST  and then re indexing the PST file(s)?

 

I have read the SCAN PST error logs and the intella logs, but neither of them seem to clear Things up for me.

 

Many thanks and best Regards,

 

mitch

 

 

[AdamS]

I know ScanPST doesn't always fix all the errors with a single run. As a general rule run Scan PST over and over until you get a 'no errors located' message, this can sometimes take 6 or more runs.

 

There is a batch version of ScanPST somewhere that you can set it to auto run until it's error free.

 

Other than that as a general rule I run Scan PST on all PST/OST files before I ingest them with Intella, more time on the front end can save bucket loads on the back

[jasoncovey]

I have processed hundreds of PSTs with Intella, and when I encounter corruption, ScanPST has succeeded in resolving the issue for me well less than 50% of the time such that the file will then process correctly in Intella.  OTOH, there has only been one case in which a PST would open properly in Outlook, yet fail to index in Intella.  From a time and practicality perspective, third-part tools have proven the most useful for me in these situations.  I have had very good luck with Kernal for PST Repair.  Vound has previously mentioned that their favorite is PST Crawler, as discussed in this thread.

 

Hope that helps!

[Chris]

Hello all,

 

From our side, I can say that we are using a software library for PST/OST access that from the ground up has been developed by forensic experts and for the purpose of forensic analysis. I suspect the way it works will be very similar to ScanPST, in that it scans the entire file for messages, incl. messages that are not linked from any indices, to improve the chance of reporting 100% of the messages when the file is corrupt.

 

Though ScanPST has the benefit of being the Outlook vendor's own tool, it is designed for email recovery, not forensic analysis. I can imagine (never tested this myself) that this could give issues with e.g. PST-specific metadata like email creation dates. What are your experiences with that?

 

I will ask our developer most knowledgeable on this subject matter to comment on our code, e.g. explain the exact difference between recovered and orphaned mails, but he's on leave now.

 

Be the way, I can think of two ways why ScanPST would increase the file size:

• PST files grow in chunks, so perhaps ScanPST allocated a lot more space and may only have used a little of it.
• I have been told that a PST/OST file contains several indices for fast access to the mails. Perhaps some of these were missing in the corrupt file and were rebuild from the others or from the entire list of emails.

In both cases the PST can be significantly bigger without Intella or ScanPST reporting any more messages.

[Guest dk_mitch]

Thanks everyone for their answers, much appreciated. Looking forward to your developers reply Christiaan.

 

In our example using SCANPST, Intella reported 150k more emails along with 500+ email containers, twice as many documents etc.

 

Also, does anyone know what the files are that appear in the folder of the PST file ?

 

I have several thousand , 99% are under 1kb in size.

 

I have asked Microsoft to explain. I somehow think that they are just scraps.

 

But due to the nature of the case, I need a definitive answer.

[igor_r]

Hello all,

 

First of all, I agree with my colleague Christiaan.

 

 

So I would like to ask the ADMIN's to explain what is done while Intella first ingests / crawls a pst file.

 

There are three steps in a process of ingesting a PST file:

1) The main tree.

This is a place where all regular items are located. Basically this is what Outlook shows.

If the main tree index can not be read then Intella will show an error. Using of ScanPST may help in this case.

 

2) Orhpan items. (<ORPHAN ITEMS>)

Apart from the main tree there are also other regular items, but they are not connected to any folder.

So they can not be seen in Outlook. We are not sure about its purpose, but we think that 

they might be left-overs from internal Outlook operations.

For example, when you edit/compose an email Outlook may save a temporary copy of this email.

 

3) Recovered items. (<RECOVERED> folder)

In 1 and 2 we talked about allocated space. But there are also unallocated data blocks.

When Outlook deletes an item, it's marked as unallocated and removed from the main tree, but the data is still there.

Intella scans all unallocated data block and tries to recover deleted items.

 

 

Now regarding using ScanPST. I think it would make sense to use it even if the PST is not corrupted. But that may not sound forensically as it modifies the original data.

As my colleague Christiaan said, it's written by Microsoft, so the tool might be able to recover some data that can't be recovered by Intella.

 

 

In our example using SCANPST, Intella reported 150k more emails

 

Are all those 150K items unique?

[Guest dk_mitch]

Hi Igor, thanks for your explanations of ORPHAN and RECOVERED, I am still waiting for a reply from Microsoft.

 

To answer your question about the 150k more emails.... I am not sure if they are all Unique, as the entire universe is 250k +. IS there a way I can determine this for you ?

 

But I have the original case and the scanpst case available for further testing and I would like to get to the bottom of this.

 

But since it is 120 GB of files, I was hoping to reproduce this with a smaller file.

 

My main concern is actually the forensic soundness of this entire process. I have never heard this topic discussed much, but maybeI am listening in the wrong Places !

 

Br, mitch

[Guest dk_mitch]

Igor, I have a partial answer to your question about the 150k emails, seen from the perspective of the ORPHANED and RECOVERED folders perspective.

In the original case, there was 11 of each. In the SCANPST version, it increased  to approximately 1670 and 2770 - I sorted them by URI once I was made aware of the issue.

 

I hope this helps and please let me know if there is more I can do to help.  I can see my level of understanding in this area is much too low.

 

Br and thanks,

 

mitch

[Chris]

To answer your question about the 150k more emails.... I am not sure if they are all Unique, as the entire universe is 250k +. IS there a way I can determine this for you ?

 

The procedure would be to query for all and folders, selecting all results and deduplicating the results table.

 

Unfortunately you have to select these folders for every individual PST file in the Location tree; there is no category in the Features facet (yet) that leads you to all recovered or orphaned items.