Jump to content

Exporting Intella Case as an Intella Case


Recommended Posts

Hi All,

 

Hoping to get some advice -- I have a case with one source (~80GB, >150,000 items) loaded into Intella and I have tagged a large number of items to be reviewed by another person at a different site.  Is there a way to export this material (ONLY the tagged material) as a separate Intella case?  

 

I've tried exporting the files in their original format but it is taking a significant amount of time and I would like to avoid having to re-index all the material anyway.  The material that is not tagged is privileged information that must be redacted, so it is important that it is not included with the source material being reviewed at the other site. 

 

What is the best way to handle this?

Link to comment
Share on other sites

Hi gcahlik,

 

In the current version there is no automatic function to export tagged items into another case. 

 

The work around is what you have already mentioned - export the items in original format, then index these items in a new case. May be others on the forum have done it a different way, and can share their workflow with us?

 

This question of privileged material has been asked before. It is likely that we will include such functionality in a future version of Intella.

 

Jon 

Link to comment
Share on other sites

Thanks Jon -- 

 

Do you have any suggestions to speed up the process?   I have several cases, ~80 GB that I would need to have exported and it's taking an impossibly long time.  Initial estimates were at 3 hours, now after 2 days of straight running, it appears to be less than 25% completed with 8 days remaining.  I thought it might be a problem with images and media; having to deal with compression and encoding, so I've tried to limit it to just documents and communications, but even that seems to be taking an incredible amount of time -- the original estimate was <1 hour for 225,000 items, now, after running for 3 hours, it is at about 55% completed with 3 more hours estimated in the export.  It did not take nearly this long to index these, why is it taking so long to export?

 

George 

Link to comment
Share on other sites

Hi George,

 

What type of system/spec to you have? Also, have you moved the evidence or case data from when you initially created the case? 

 

These speeds do not sound right. It could be an I/O issue. Are the case and evidence drives local to the system (e.g. mounted within the system). USB connected drives can be a factor and we recommend not to use them.

 

Jon

Link to comment
Share on other sites

  • 2 weeks later...

Hi Jon,

 

I've run it on an Mac Pro, Xeon E5-2697 w/ 64 GB RAM and I've run it concurrently with a i7 2860 with 16 GB of ram.   All of the sources are on stored on the internal SATA drive, no external drives.  I'm having problems with both platforms but at the moment I have it on my i7;  it's having problems with just one source -- it's about 80 GB with around 1.6M items.    The cases were created by another office, I have just copied them to the internal drives of my review computers. 

 

What I ended up trying is exporting results by file type facet -- exporting them separately (ie. all the communications in a batch, all of the documents) -- the hold up are the images.  I had no problem processing the ~400,000 documents and communications -- it completed this task in under 6 hours, but with the images it is choking.   I further began breaking it down by image type.  I exported ~70,000 gifs with in about 4 hours, but I went to do 60,000 wmf files and it has been cranking for 10 hours now and it's just 75% done and it states that it has 4 hours left, but I know that's in accurate because that's what it said 3 hours ago.   I haven't even touched the JPGs and PNGs, which account for about 500,000 items total. 

 

I have the Mac Pro at another location exporting the dataset as a whole and last check (Tuesday 8/1), after 10 days of countinuous operation, it's still less than 67% complete -- the progress meter says 7 more days, but like I said, that's inaccurate. 

 

George

Link to comment
Share on other sites

I killed off the WMF export; there were still about 12,000 items remaining after 10 hours of exporting and at this point, it was only exporting one file every 10-15 seconds.    I deleted the exported files and went back and started over -- this time, I further broke it down and exported only 10,000 of the 70,000 wmf files and it completed the export in 10 minutes.  It makes me feel like this is some sort of memory leak. 

Link to comment
Share on other sites

Hi Jon,

 

Yes, I am using version 2.0.1.1.   

 

I've been working at doing this all day today, exporting them in portions; It took about 4 hours to export exactly 50,000 JPEGs.  The subsequent exports were cut down to 25,000 JPEGS and it has taken about 1 hr 15 minutes.   I do not have any other programs running in the background and for all exports, it starts off relatively fast, processing around 1500 items a minute, but then it gets progressively slower and slower.  

 

George

Link to comment
Share on other sites

Hi Jon,

 

I'm starting to think this is some sort of IO issue, as you had asserted earlier -- but it's still strange none the less.  

 

I tried a few experiments -- these were conducted on our Mac Pro.  

 

First I selected 10,000 items to be exported, all of the same file format (png images).  They were going to be exported into one folder, no directories.  

 

Here are the time stamps per 1,000 items processed.

 

1,000:  0:06  (6 seconds)

2,000:  0:17  (11 seconds)

3,000:  0:34  (17 seconds)

4,000:  1:01  (27 seconds)

5,000:  1:34  (32 seconds)

6,000:  2:18  (46 seconds)

7,000:  3:12  (54 seconds)

8,000:  4:05  (53 seconds)

9,000:  4:57  (42 seconds)

10,000: 5:50 (53 seconds)

 

So the next experiment I had was to start exporting them in 5,000 item chunks.  I had them originally exporting into individual directories.  So chunk 1 went into Directory 1 and chunk 2 into Directory 2 and so forth.  All exports were under 2 minutes.

 

I then had them export into the same directory, here were the results, total time for each export:

1st 5k chunk: 1:47

2nd 5k chunk: 5:47

3rd 5k chunk:  8:24

4th 5k chunk:  12:10

 

So, for whatever reason, when I export them into the same directory, each progressive one takes longer and longer.   Obviously this is arelatively small sample size, but it's still hard to wrap my head around it.

Link to comment
Share on other sites

There is definitely some underlying other issue here as I've regularly exported out huge data sets (300GB +) and never had any take longer that 7 hours or so.

 

In terms of case size you mentioned 80GB as the source, so how much tagged data are you exporting size wise? I may be stating something you already know, but if you are able to do something with you hard drive configurations you can get huge speed improvements.

 

By that I mean 1 drive for original source data, a different drive for the resulting case data and a third drive to use for optimization. Again when exporting data it's always going to be quicker if the exported data is going to a new drive that is not in use for anything else...but not always possible I know.

 

My process in this case would be:

  1. Export tagged items by tag (you end up with a single folder for each tagged item) if relatively few tags OR
  2. If too many tags for that approach export all as a single data set THEN
  3. Export MD5 lists for each tag
  4. Review in other location sets up new case and indexes data
  5. In the case of individual folders per tag they can immediately start review OR
  6. Use the MD5 list to auto search and tag files to duplicate the tags, then begin review.

Of course by far the simplest method would be to use Connect and setup a remote review for the second site, this would involve no exporting or retagging and the LPP stuff would be excluded...but of course it does involve purchasing a Connect license :)

 

Side issue, I tried to update my photo for my profile and it appears that the ability to upload a photo is broken :(

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
 Share

×
×
  • Create New...